Is the Web killing the Impact Factor?

Nicholas Genes (@nickgenes) just tweeted a link to an article in the Atlantic titled Thanks to the Web, Even Scientists Are Reading for the Articles. This article highlights findings from this recent paper:

• Lozano, G. A., Larivière, V. and Gingras, Y. (in press) The weakening relationship between the Impact Factor and citation rates in the digital age. Journal of the American Society for Information Science and Technology [free fulltext pdf]

This is the abstract:

Historically, papers have been physically bound to the journal in which they were published but in the electronic age papers are available individually, no longer tied to their respective journals. Hence, papers now can be read and cited based on their own merits, independently of the journal’s physical availability, reputation, or Impact Factor. We compare the strength of the relationship between journals’ Impact Factors and the actual citations received by their respective papers from 1902 to 2009. Throughout most of the 20th century, papers’ citation rates were increasingly linked to their respective journals’ Impact Factors. However, since 1990, the advent of the digital age, the strength of the relation between Impact Factors and paper citations has been decreasing. This decrease began sooner in physics, a field that was quicker to make the transition into the electronic domain. Furthermore, since 1990, the proportion of highly cited papers coming from highly cited journals has been decreasing, and accordingly, the proportion of highly cited papers not coming from highly cited journals has also been increasing. Should this pattern continue, it might bring an end to the use of the Impact Factor as a way to evaluate the quality of journals, papers and researchers.

In essence, it appears that scientists no longer read journals, instead they read articles. It seems that the web is facilitating the dissemination of published articles, regardless of the journal from which they originate.

This graph, taken from the paper, shows what has happened in the medical and natural sciences:

On the y-axis is a co-efficient indicating the strength of the relationship between a journal’s impact factor and the number of articles read, with year on the x-axis. Since the 1990s, journal impact factor has had a progressively weaker relationship to the number of articles read.

Can you hear that ringing sound? It could be the death knell for the impact factor…

So, what will we use instead? Lozano, Larivière, and Gingras have a novel suggestion among their numerous criticisms of the impact factor:

As the relationship between paper citation rates and [impact factor] continues to weaken, and as more important papers increasingly appear in more diverse venues, it will become even less justifiable to automatically transfer a journal’s reputation and symbolic capital on to even its most recently published papers. This should force a return to direct assessments of paper quality, by actually reading them.