econintersect.com
       
  

FREE NEWSLETTER: Econintersect sends a nightly newsletter highlighting news events of the day, and providing a summary of new articles posted on the website. Econintersect will not sell or pass your email address to others per our privacy policy. You can cancel this subscription at any time by selecting the unsubscribing link in the footer of each email.



posted on 15 February 2016

Are Citation Rates The Best Way To Assess The Impact Of Research?

from The Conversation

-- this post authored by Simon Chapman, University of Sydney

The United Kingdom's Medical Research Council (MRC) - the equivalent of Australia's National Health and Medical Research Council - recently released its 2014-15 economic impact report, details of which make for interesting reading.

Since 2006, research funded by the MRC resulted in more than 94,000 publications, 63,000 of which (67%) were peer-reviewed.

The traditional starting point for considering the scientific impact of research are its citations. This is how many other research papers and editorials subsequently cite a given paper into the future or in a given number of years since publication.

Different numbers of researchers are involved in different fields of research. So when a relatively small number of scientists work in a study area, even if they write a spectacularly important paper, it can still receive a small fraction of the citations a comparably important paper receives in an area where more researchers are involved.

It would be unreasonable and misleading to conclude that a leading researcher in a small field had less scientific impact than one in a big field. Also, as can be seen from this table, a paper that has been published for a short time has naturally had less time to be cited than one that has been out for longer.

Normalised citation impact

For these reasons, citation analysts use the normalised citation impact (NCI) to adjust citation volumes in different fields. This allows them to be compared in analyses of entire research funding schemes, national research and international activity.

The MRC report provides a graph showing the NCIs for all MRC-funded research publications for the sample period 2006-13. The average NCI citation for all papers in health and medical fields across these eight years was a desultory 2.08. And this was more than twice the world average.

The report notes that of more than six million papers, more than a fifth (21%) had not been cited, while only 3% of the MRC funded research had no citations to date.

What counts as a high and very high citation in the MRC data is also interesting. "Highly cited" means a paper with just four or more cites and "very highly cited" is anything with eight or more.

The situation in Australia is more opaque. A 2013 report provides lots of comparative data that show Australian health and medical researchers punch well above our weight compared to other nations and our small population.

But nowhere could I find comparable data to that provided by the MRC report. However, the NHMRC report notes:

The citation distribution among publications is very skewed. While very few publications achieve high citation counts, a vast majority receive very few or no citations at all.

Citation is of course only a measure of interest in your work by other researchers.

Journals that provide all or some of their papers as open access to anyone, often have daily updated counters showing the number of readers who have been to the publication. Not all of these will have read it and only land on it in a search for something else, so the data exaggerate actual readers.

My most read paper is one looking at the incidence of gun massacres and deaths ten years after the 1996 Australian gun law reforms.

It received some 136,946 views since 2006, 82,312 of which were in December 2012 in the aftermath of the Sandy Hook school shootings in the United States. I tweeted a link to the paper, that saw it go viral.

Ironically, the paper wasn't funded by any research grant.

Altmetric

A metric increasingly being used by researchers since 2013 to demonstrate wider interest in their work is Altmetric.

The Altmetric score provides an index of the extent to which a research paper is being circulated and discussed on social media and covered in the news media.

My 2006 firearms paper has a stratospheric Altmetric score of 2,118. The 100 highest Altmetric scoring papers across all research fields in 2015 are listed here. Had mine been published in 2015 with the same attention it has received it would have had the seventh highest Altmetric score that year.

The MRC report provides data on a range of social impacts that go well beyond metrics. The data assesses the direct impact of research on outcomes that may have social and economic impact.

These include:

  • The development of more than 4,400 instances of influence on policy and practice - 416 new in 2014, including 472 citations in clinical guidelines.

  • The development of more than 1,000 products and interventions - 126 new in 2014.

  • The creation or growth of 88 companies - seven new in 2014.

  • Approximately 1,081 patents - 37 filed or granted in 2014, with discoveries related to 246 (23%) of these patents already licensed worldwide.

In work we published in 2015 examining the "impact in society" of intervention research in health and medical research funded by the NHMRC, we found 38% of research projects could demonstrate some level of social or health service impact.

We investigated the characteristics of those projects that demonstrated impact and compared them to those that didn't.

Our study indicated that sophisticated approaches to intervention development, dissemination actions and translational efforts, are actually widespread among experienced researchers, and can achieve policy and practice impacts.

However, it was the links between the intervention results, further dissemination actions by researchers and a variety of contextual factors after the research that ultimately determined whether a study had policy and practice impacts.

Given the complicated interplay between various factors, there (alas) appears to be no simple formula for determining which intervention studies should be funded in order to achieve optimal policy and practice impacts.

The judgement of which research applications should be funded is, and is likely to remain, a very inexact science.

The ConversationSimon Chapman, Emeritus Professor in Public Health, University of Sydney

This article was originally published on The Conversation. Read the original article.

>>>>> Scroll down to view and make comments <<<<<<

Click here for Historical News Post Listing










Make a Comment

Econintersect wants your comments, data and opinion on the articles posted.  As the internet is a "war zone" of trolls, hackers and spammers - Econintersect must balance its defences against ease of commenting.  We have joined with Livefyre to manage our comment streams.

To comment, using Livefyre just click the "Sign In" button at the top-left corner of the comment box below. You can create a commenting account using your favorite social network such as Twitter, Facebook, Google+, LinkedIn or Open ID - or open a Livefyre account using your email address.



You can also comment using Facebook directly using he comment block below.





Econintersect Contributors


search_box

Print this page or create a PDF file of this page
Print Friendly and PDF


The growing use of ad blocking software is creating a shortfall in covering our fixed expenses. Please consider a donation to Econintersect to allow continuing output of quality and balanced financial and economic news and analysis.


Take a look at what is going on inside of Econintersect.com
Main Home
Analysis Blog
The Problem With Obamacare Is That It Did Little To Reduce Overall Healthcare Spending
Joan Robinson’s Critique of Marginal Utility Theory
News Blog
Early Headlines: Russia Hacked GOP, Trump To Drain Energy 'Swamp'?, New Sec'y Of State Candidate, India IP Shrinks, India Has World's New Largest Solar Plant , China GDP Hides Volatility And More
Most Coup Attempts In Recent Years Have Failed
The Global Cost Of Diabetes
The Universities Churning Out The Most Billionaires
Five Amazing Ways Plants Have Created New Technologies
Where U.S. Weekly Wages Go The Furthest
What We Read Today 09 December 2016
How To Stop Using Filler Words Like Um And Uh
02 December 2016: ECRI's WLI Growth Index Improvement Continues
Preliminary December 2016 Michigan Consumer Sentiment Highest Since Early 2015
October 2016 Wholesale Sales Improved
Rail Week Ending 03 December 2016: Finally A Positive Month
November 2016 CBO Monthly Budget Review: Total Receipts Up by 1 Percent in the First Two Months of Fiscal Year 2017
Investing Blog
The New Art Of Utility Investing
Investing,com Weekly Wrap-up 09 December 2016
Opinion Blog
Trickle-down Economics, Trump Edition
Looking At Everything: Trump's $1 Trillion Infrastructure Plan
Precious Metals Blog
Silver Prices Rebounded Today: Where They Are Headed
Live Markets
09Dec2016 Market Close: Wall Street Closes On A New High, Trump Sugar High, Crude Prices Testing Resistance, US Dollar Melts Higher
Amazon Books & More






.... and keep up with economic news using our dynamic economic newspapers with the largest international coverage on the internet
Asia / Pacific
Europe
Middle East / Africa
Americas
USA Government



Crowdfunding ....






























 navigate econintersect.com

Blogs

Analysis Blog
News Blog
Investing Blog
Opinion Blog
Precious Metals Blog
Markets Blog
Video of the Day
Weather

Newspapers

Asia / Pacific
Europe
Middle East / Africa
Americas
USA Government
     

RSS Feeds / Social Media

Combined Econintersect Feed
Google+
Facebook
Twitter
Digg

Free Newsletter

Marketplace - Books & More

Economic Forecast

Content Contribution

Contact

About

  Top Economics Site

Investing.com Contributor TalkMarkets Contributor Finance Blogs Free PageRank Checker Active Search Results Google+

This Web Page by Steven Hansen ---- Copyright 2010 - 2016 Econintersect LLC - all rights reserved