Better Mousetraps and Railroad Gauges

Better? This image courtesy of Wikimedia Commons, a freely licensed media file repository.

A recent article in Nature (“Computing giants launch free science metrics,”   2 Aug 2011) describes the launch of two major new (and free!) tools for researchers to use to track citation statistics, — who’s citing whose publications, a widely used metric for evaluating scholarly impact and to track research trends.   “Mapping the landscape of science is about to get easier than ever before. Google and Microsoft are rolling out free tools that will enable researchers to analyse citation statistics, visualize research networks and track the hottest research fields.  … “ (read more).

For many years the gold standard for citation metrics has been the Web of Science citation databases, produced by ISI (now Thomson Reuters).   The early, print volumes of the Science Citation Index (Web O’ Science’s predecessor) occupied miles of shelf space in libraries and came with their own magnifying glass to help you make sense of the tiny print and cryptic abbreviations.   (This may be apocryphal, but it’s my story and I’m sticking with it.)

Web of Science (I’ll use the current name for convenience) has enormous competitive advantages in this specialized market:  citation data back to 1946 in many disciplines especially in the sciences (with a limited subset even reaching back to 1900); exhaustive, if not perfect, control of variants (author name variants, article citation variants – imagine how tedious this used to be to normalize, before computer-assisted indexing); and extensive coverage of major journals in virtually all disciplines.    So!  A dominant force, not easily challenged.  (Though a recent entrant to the citation combat lists, Elsevier’s Scopus, has its adherents as well.)

However, Web of Science does have some serious limitations, especially for non-science disciplines, in that it tracks only (pretty much only) citations from journal literature, so that books and conference proceedings, which are critically important publication formats in the humanities, social sciences and engineering, for example, are omitted, resulting in significant undercounts in certain fields.   And as a side note, Web of Science is a costly resource (at the risk of being indelicate, about as much, say, as a decent condo, – each and every year).     Some institutions and libraries, especially in poorer countries, are not able to license these giants and have had to do without.

The new citation services launched by Google and by Microsoft are important for these reasons, – they are free (on the rising tide of high-caliber open access resources) AND they capture a much wider percentage of the scholarly literature than does Web of Science (especially Google, whose product, Google Scholar Citations, builds on the enormous Google Scholar and Google Books databases).   So, they are, perhaps, or promise to be … a better mousetrap.   Also cool visualization tools and charts – read the article.

However, one final point about citation metrics and tools – as long as citation statistics are used  comparatively to evaluate scholarly impact – for promotion and tenure decisions, for assessing grant proposals, for any of the many uses that these overworked metrics are put to, — then there is a compelling need for a common standard, regardless of how many better mousetraps there may be.   We may end up with our version of the 19th century Gauge Wars (also see Railroad Gauge or Track Gauge).    Still, interesting times!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s