The “Core Influence” of Journals in Sociology Revisited
by Michael Patrick Allen,
Washington State University
Thirteen years ago, I proposed an objective measure of journal quality that I termed “core influence” (see November 1990 Footnotes). It measured the “influence” of a journal within the discipline in terms of the number of times the average article published in that particular journal would eventually be cited in the three “core” journals of the discipline: American Sociological Review, American Journal of Sociology, and Social Forces.
This measure attracted more attention than I had anticipated. The results published in the Footnotes article have been cited by researchers studying sociology as a scientific field as well as by sociologists who were in the process of being evaluated for promotion and tenure. However, the empirical results reported in that article have become increasingly dated. When I first proposed this measure, I assumed that the relative influence of the journals within the discipline would not fluctuate dramatically in the short run. But I also understood that the influence of specific journals might rise or fall over the longer run. Consequently, I recently decided that the time had come to update and expand my original analysis. The methods employed to create the new core influence scores are identical to those employed 13 years ago. The data used to construct the current measures of journal influence were obtained from the Journal Citation Report, an electronic bibliographic database compiled by the Institute for Scientific Information.
I will not reiterate here the details of the argument offered in defense of this measure of journal quality except to say that, basically, the core influence of a journal is defined as the number of times that articles published in that journal have been cited by the three core journals in sociology in a given year divided by the number of articles published by that journal in that same year. If the number of articles published by a journal remains relatively constant over time, this ratio provides a reasonable estimate of the number of times that the average article published in that journal will eventually be cited in the three core journals. For example, if all of the articles published in American Sociological Review, American Journal of Sociology, and Social Forces cite articles published in Administrative Science Quarterly an average of 79 times a year and if Administrative Science Quarterly publishes an average of 18.7 articles a year, then the core influence score of Administrative Science Quarterly is 4.23 (i.e., 79/18.7). In other words, we can expect that the average article in Administrative Science Quarterly will be cited a total of 4.23 times in these three core journals in sociology over time.
Table 1 presents in rank order the core influence scores for the 90 journals with the largest average scores for the three-year period from 1999 to 2001. This table also presents the core influence scores for the three-year period from 1986 to 1988 for those journals that were also included in my original study along with their original ranks. A comparison of the journals that were ranked in both time periods indicates that their relative scores are quite stable. Indeed, the correlation between the core influence scores for those 53 journals that were included in both studies is 0.968.
The stability of these scores over these 13 years suggests that this measure of journal influence is highly reliable. Despite the overall pattern of stability, it is important to note that the core influence scores for several journals have changed appreciably during this period. These shifts in the core influence scores of particular journals may be partly attributable to changes in the popularity of specific specialty areas in sociology. They may also be attributable to changes in the editorial policies of those journals.
In comparing the journals listed in Table 1, there is a temptation to focus on the rank order of these journals rather than on the magnitudes of their core influence scores. However, it is important to realize that the magnitude of the score associated with each journal is more important than its rank. In general, the differences between the most influential journals and those below them, in terms of how often they are cited in the core journals, are substantial. At the same time, the differences between most of the less influential journals, especially those with roughly comparable scores, are often inconsequential. For example, the difference in the core influence of Sociological Perspective, with a score of 0.77, and Sociological Quarterly, with a score of 0.73, corresponds to a difference of only four citations in the three core journals over a period of three years. Moreover, it is worth noting that the core influence scores of most journals have increased over time. In short, the articles published in the core journals are citing more articles than they did 13 years ago. This trend may be a function of the increased availability of electronic bibliographic databases.
Finally, it should be evident that it is possible to decompose the core influence score of any journal into the components attributable to each of the three top journals. The components of the core influence measure attributable to American Sociological Review, American Journal of Sociology and Social Forces, respectively, are also presented in Table 1. For example, this table indicates that an article published in Administrative Science Quarterly, which has a core influence score of 4.23, can expect to be cited 2.45 times in American Journal of Sociology, 1.20 times in American Sociological Review, and 0.59 times in Social Forces.
Indeed, these data confirm that each of the three core journals of the discipline publishes articles that tend to cite certain specialty journals more than others. For example, American Sociological Review cites more articles in Demography than does Social Forces. Similarly, American Journal of Sociology cites more articles in Administrative Science Quarterly than does American Sociological Review. Finally, Social Forces cites more articles in Journal of Scientific Study of Religion than does American Journal of Sociology.
Several caveats noted in the original discussion of this measure of core influence deserve to be reiterated. First, the accuracy of this measure depends on the assumption that the number of articles published by a journal each year has not changed appreciably over time. Although most of the citations employed in these computations are to relatively recent articles, other citations are to articles published several years ago. If a journal currently publishes more articles each year than it did previously, the procedures employed in this analysis will underestimate the core influence of that journal. The converse is also true. This fact also implies that newer journals are at a disadvantage in comparison to established journals with respect to the number of citations they receive. Established journals have a larger inventory of articles available for citation than newer journals. Second, the validity of this measure of core influence rests on the assumption that American Sociological Review, American Journal of Sociology, and Social Forces are, in fact, the core journals of the discipline.
In order to confirm the validity of these core influence scores, a separate analysis was conducted of the network of citations among 64 leading journals within the discipline between 1999 and 2001. The centrality score of each journal was obtained from the first eigenvector of the asymmetric citation matrix after it had been adjusted for the number of self-citations and the number of articles published by each journal. This network analysis confirms that American Sociological Review, American Journal of Sociology, and Social Forces are the three most central journals of the discipline in that order. Moreover, the correlation between the core influence scores and the centrality scores for the 58 journals included in both analyses is 0.959. In short, this network analysis of the centrality of the leading journals in sociology confirms the validity of the core influence scores. The main disparities between the two measures involve the leading journals within major specialty areas, such as Journal of Marriage and the Family and Criminology, which have comparatively high centrality scores because they are cited frequently by other journals in those areas. Given their apparent reliability and validity, core influence scores are probably preferable to centrality scores because they are highly interpretable and relatively easy to compute.
When I first proposed this measure of journal quality, I expressed misgivings about the possible misuses of such a measure. I stated that “the very existence of objective measures of journal quality may discourage those who must evaluate the work of sociologists from taking the time and effort required to assess this work on its own intrinsic merits or even in terms of its subsequent impact on others in the discipline.” Nevertheless, there are many situations in which it is difficult to evaluate directly the quality of articles. One of these situations occurs whenever the members of a search committee must evaluate the publication records of a large number of applicants. This situation may also arise whenever junior faculty members are evaluated for tenure and promotion. It is difficult to assess the eventual impact of recently published work. Moreover, the discipline has become so specialized that it often difficult for even established scholars to make informed judgments about the originality and significance of work outside their areas of expertise. At the very least, these measures of core influence tells us something about the quality of the journals in which this work is published.
The author is indebted to Lowell Hargens for his assistance and advice. The results of the centrality analysis of the leading journals in sociology mentioned in the text maybe obtained at www.wsu.edu/~allen/centrality.pdf.