Sunday, December 16, 2012

The cost does not count

Researchers, like, say, teachers or bricklayers, can be bad, average, good or excellent. One criterium to tell among the tens of thousands of scientists on our planet is analyzing their scientific production. And one way to do so is the -already famous- H-index. Wikipedia defines H-index as follows: "A scientist has index h if h of his/her Np papers have at least h citations each, and the other (Np − h) papers have no more than h citations each".
In other words, a scientist with a h-index of 8 (that's me) has eight papers with eight or more citations. It seems fair, as it takes into account both production (number or papers) and their impact (times they have been cited by other reports). In fact it is far from being fair. First, the index strongly depends on the age. The older you are the longer is the period during which your papers can be cited. If I die today, my h-index will continue to grow as my articles are being cited post-mortem. Articles need time to be cited and young researchers have young articles too. There are many more criticisms to h-index (see http://en.wikipedia.org/wiki/H-index). But I would like to highlight one reason that, to my knowledge, has not been stressed up to date among the cons of using h-index as an evaluation parameter: funding.
The index does not take into account how much the published research costed. An article may be the result of a 100,000 euros project, but a similar quality research paper may have only costed, for example, 10,000 euros. This means that with the first sum it would be possible to publish 10 papers instead of only one. In my view, taking into account the ratio between bulk production/citation (used to calculate the H-index) and funding (F) makes a more realistic proxy for productivity (P).  Therefore, I propose an alternative parameter to h-index, P-index, which as you can imagine, is calculated as simply as this:
P-index=H-index/F
What do you think?

How do we measure research quality?

1 comment:

  1. I think it is difficult to evaluate the production. I guess h-index is a fast and rough approach, as making an index more accurate will make it more difficult to obtain the information needed to calculate it. Should the cost of every research be published in order to be able to calculate the P-index?. We are also leaving behind the age of the article (time should affect negatively, I think).

    ReplyDelete