Impact factors

Display of impact factors in Journal Citations Reports (JCR)

To browse other articles on a range of HSL topics, see the wiki index.

Introduction

See also Altmetrics , Author impact metrics & Scholarly publishing and communication

An impact factor (IF) is a significance measure of a journal within a discipline, subject area or profession. An IF is determined by averaging the number of citations a journal receives and the average number of times that articles within the journal are referenced by other articles. First conceived by Eugene Garfield in the 1950s, IFs came into widespread use in the 1970s and are well-established in the 21st century. (Garfield's company is now Thomson Reuters, a worldwide US-based publisher.) In fact, many university faculty and researchers use IFs to determine their influence and to reveal relationships between them, other writers and their scholarship.

The most common way to find IFs is to search the Social Science Citation Index, Science Citation Index and/or Arts and Humanities Citation Index. These tools help researchers track down citations across fields (see Thomson Reuter's Web of Science - WoS). WoS is also known as the Web of Knowledge. Although some IFs can be determined using Google scholar, Jacso criticizes it for inflated citation counts and inaccuracies. One recent project uses GS to determine author influence using a Tenurometer.) A few of the top journals in medicine are the New England Journal of Medicine, British Medical Journal, Journal of the American Medical Association (JAMA) and Lancet. IFs have a significant impact in promotion and tenure, publishing cycles, funding and/or grants.

Beyond impact factors

• Immediacy index - average citation number of an article in that year;
• Journal cited half-life - median age of articles cited in Journal Citation Reports annually. For example, if a journal's half-life is 5, that means citations from 2001-2005 are 50% of all citations from that journal in 2005.
• Aggregate impact factor - calculated by # of citations to all journals in category and number of articles from all the journals in the subject category.

Impact factors refer to journals not specific articles or authors. The number of citations for individual articles is called citation impact. It is possible to measure the IF of journals in which a particular person has published articles; a controversial but widespread use. Garfield warned about the "misuse in evaluating individuals" for there is "wide variation from article to article within a single journal".

Calculation

IF is calculated based on a three-year period, and the average number of times papers are cited up to two years after publication. For example, the 2008 impact factor for a journal would be calculated as follows:

A = number of times articles published in 2008-9 were cited in indexed journals during 2009

B = number of articles, reviews, proceedings or notes published in 2008-9

2009 impact factor = A/B

(Note: 2009 impact factors are published in 2010 and calculated after all 2009 publications are reviewed.)

Another way to see an IF is that where a journal is cited once each article published has an IF of 1; there are no articles to be averaged just one article. Thomson Reuters excludes certain types of articles (i.e. news items, correspondence, and errata) from the denominator of the IF. New journals indexed from inception receive IFs after two years' of indexing; citations prior to Volume 1, and number of articles published prior to Volume 1, are known as zero values.

Controversies of impact factors

IFs are useful in comparing different journals. For example, a sponsor of research may want to compare the productivity of its projects and their impact. At times, an objective measure of the importance of publications is needed and the impact factor (or number of publications) is the only one available. It is important to remember that scholarly disciplines can have different publication and citation practices, which will affect the number of citations and how quickly articles in the subject reach their peak citation counts. In all cases, it is relevant to consider the rank of the journal in a category of its peers, rather than the raw Impact Factor value. Impact factors are not infallible measures of journal quality. For example, it is unclear whether the number of citations a paper garners measures its actual quality or simply reflects the sheer number of publications in that particular area of research and whether there is a difference between them. Furthermore, in a journal which has long lag time between submission and publication, it might be impossible to cite articles within the three-year window. Indeed, for some journals, the time between submission and publication can be over two years, which leaves less than a year for citation. On the other hand, a longer temporal window would be slow to adjust to changes in journal impact factors. Thus, The impact factor is appropriate for some fields of science such as molecular biology, it is not for such subjects with a slower publication pattern. (It is possible to calculate the impact factor for any desired period, and the web site gives instructions.)

Why IFs are useful

• Used for promotion and tenure;
• Wide international coverage at ISI;
• Web of Knowledge indexes 9000 science and social science journals from 60 countries.
• Results are widely (though not freely) available to use and understand;
• An objective measure, with a wider acceptance and reliability than the alternatives;
• One alterative measure of quality is "prestige" - a rating by reputation, which is very slow to change, and cannot be quantified or objectively used. It merely demonstrates popularity.

Drawbacks of impact factors

• Inadequate international coverage; Web of Knowledge indexes journals from 60 countries, but coverage is uneven. Few publications from languages other than English are included, and few from less-developed countries.
• Numbers of citations to papers in particular journals do not measure quality or scientific merit.
• Journals with low circulation, regardless of scientific merit of content, will never obtain high impact factors.
• Since defining the quality of an academic publication is problematic, involving non-quantifiable factors, such as the influence on scientists, assigning a value is difficult.
• Time factors for citations are too short. Classic articles are cited frequently even after several decades.
• The number of researchers, average number of authors and nature of results in different research areas, make impact factors between different groups of scientists difficult.
• Generally, medical journals have high impact - a fact accepted by publishers but this does not mean they are useful in other fields--such a use is an indication of misunderstanding.
• By counting frequency of citations per article and disregarding the prestige of the citing journals, IF becomes merely a metric of popularity not prestige.

Manipulation of impact factors

A journal can adopt editorial policies that increase its impact factor. They may not involve improving the quality of published science as journals sometimes may publish a larger percentage of reviews. While some articles are uncited after 3 years, nearly all review articles receive one citation within three years raising the impact factor of the journal. Thomson ISI gives directions for removing these journals from calculations. For researchers or students with familiarity of the field, review journals are obvious. Editorials in a journal do not count as publications. However when they cite published articles, often articles from the same journal, those citations increase the citation count for the article. This effect is hard to evaluate for distinguishing between editorial comment and short articles is not obvious. "Letters to the editor"" might refer to either class. Editors of journals may encourage authors to cite articles from that journal. The degree to which this affects citation count and impact factor must be examined. Most of these factors are discussed along with ways for correcting the figures for these effects if desired. It is normal for articles in a journal to cite its own articles for those are the ones of the same merit in the same field. If done artificially, the effect will be significant for journals with the lowest citations and affect placement but only at the bottom of the list.

Skewness

Eighty-nine (89%) percent of citations of individual papers in Nature were generated by 25% of its papers. The most cited Nature paper in 2002−03 was the mouse genome. It represents the culmination of great work but is inevitably an important point of reference rather than an expression of unusually deep insight. It has received more than 1,000 citations. In 2004, it received 522 citations. Our next most cited paper from 2002−03 (about the yeast proteome) received 351 citations. Only 50 out of the roughly 1,800 cite-able items published in those two years received more than 100 citations in 2004. The great majority of our papers received fewer than 20 citations. This emphasizes that impact factors refer to the average number of citations per paper. Most papers published in a high impact journal will be cited fewer times than the impact factor suggests, and some will not be cited at all. The journal impact factor should not be used as a substitute measure of the impact of individual articles in the journal.

Alternative metrics

In 2006, Bollen, Rodriguez and Van de Sompel proposed using Google PageRank to distinguish the "quality" of citations and improve Impact Factor calculation.

   Impact Factor              PageRank            Combined
 1 52.28 ANNU REV IMMUNOL     16.78 NATURE        51.97 NATURE
 2 37.65 ANNU REV BIOCHEM     16.39 J BIOL CHEM   48.78 SCIENCE
 3 36.83 PHYSIOL REV          16.38 SCIENCE       19.84 NEW ENGL J MED
 4 35.04 NAT REV MOL CELL BIO 14.49 PNAS          15.34 CELL
 5 34.83 NEW ENGL J MED        8.41 PHYS REV LETT 14.88 PNAS
 6 30.98 NATURE                5.76 CELL          10.62 J BIOL CHEM
 7 30.55 NAT MED               5.70 NEW ENGL J MED 8.49 JAMA
 8 29.78 SCIENCE               4.67 J AM CHEM SOC  7.78 LANCET
 9 28.18 NAT IMMUNOL           4.46 J IMMUNOL      7.56 NAT GENET
10 28.17 REV MOD PHYS          4.28 APPL PHYS LETT 6.53 NAT MED

The table shows the top 10 journals by Institute for Scientific Information Impact Factor, PageRank, and a modified system that combines the two (based on 2003 data). Nature and Science are generally regarded as the most prestigious journals, and in the combined system they come out on top. That the New England Journal of Medicine is cited even more than Nature or Science might reflect the mix of review articles and original articles that it publishes. It is necessary to analyze the data for a journal in the light of a detailed knowledge of the journal literature.

Author impact metrics

See also Author impact metrics