CS Impact Factors are ‘Pants’

To clarify, ‘pants’ is a British adjective meaning ‘rubbish’. And what I mean to say is that Impact factors for the broadly based CS journals are inaccurate and should not be trusted as a mark of academic progression of ranking. Why? Well let me tell you…

Impact factors are a measure of a journals ‘impact’ on a field and therefore marks the prestige of the papers published within them. Impact factors range from 0.5 and below for bad journals to 1.5-4 for good HCI journals, to 30-36 for the likes of ‘nature’ and ‘science’ with the top rated journal in the world being ‘CA – A Cancer Journal for Clinicians’ with 101. You can find a more ‘Bio’ view on these with O’Really’s Impact Factor Boxing.

However, Impact factors via ISIKnowledge are generated from a citation count of an internal corpus/index of journals held by them; journals only but no conferences. Now journals are fewer and easier to index than conferences and in most disciplines conferences are not ranked for academic work, so don’t matter – but they matter big time for CS. And so these conference citations are not taken into account. In this case the impacts are just wrong. Indeed, thanks to the great effort of Antti Oulasvirta’s Updated for 2012: ‘Ranking of HCI conferences based on citations per paper‘ for conferences we have a nice list of HCI venues ranked by citation. But lets look at Journals (thanks to Markel Vigo for these):

  • Computers in Human Behaviour (social sciences edition): 2.2
  • HCI journal: 1.4
  • UMUAI: 1.4
  • Interacting with Computers: 1.2
  • IJHCS: 1.1
  • ACM TOIS: 1
  • BIT: 1
  • International Journal of HCI: 0.9
  • ACM TWEB: 0.8
  • ACM TOIT: 0.8
  • ACM TOCHI: 0.8
  • New Review of Hypermedia: 0.5
  • WWW: 0.5
  • UAIS: 0.3

Do we really think that ACM TOCHI is 0.8 – an OK/mediocre venue – no. Or that the HCI Journal at 1.4 is a good but not outstanding venue?

So why is this – missing conference citations, and other citations from the ISI corpus/index – well I’d suggest downloading Publish or Perish and run the journal citations section (which uses Google Scholar) to produce metrics including conference citations of a journal article – oh then we get a more accurate picture.

You also might to look at these two other interesting posts, one from Stephen Curry and another from Bjorn De Sutter and Aäron Van Den Oord; (magazine view) both of which I came across after writing this post.


One thought on “CS Impact Factors are ‘Pants’

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s