Wednesday, September 19, 2012

Proving, Recognising and Measuring Impact

Over the last two days of the EGI-Technical Forum, we've been extremely busy interviewing speakers, but I have managed to dip into a couple of sessions related to the challenge of measuring impact.

Yesterday's session on Scientific Publications Repository workshop highlighted the increasingly weighty issue of where and how to store scientific outputs (i.e. papers and associated data) from EU projects.  The important subject of acknowledgement of e-infrastructures was realised. It is apparent that while many researchers depend on increasing computational capacity most would not be able to identify the provider.  Scientists are interested in disseminating their achievements so while they might refer to a supercomputer or grid when publishing their results they generally do not specify the exact nature of the resources. But the fact remains if the end users of grids are unaware, then it is impossible for others to recognise the contribution these resources are making to scientific literature. With EC policies now in place to develop Open Access to research results, the session provided a perfect opportunity to explore the best ways to provide access and to appropriately acknowledge these resources.

Fortunately, there are projects examining the issue of harvesting this information. During the session, we heard from Pasquale Pagano, who described the OpenAIRE plus project. Their project is collecting and linking EU publications in an open access repository, which will give researchers both access to research material and valuable statistics to measure their impact.  Pagano gave us a glimpse into data that will be available e.g. publications per project, statistics for areas/institutions, publications (see slides). There are already 26,000 FP-7 publications from 5,000 projects (>10K Open Access). How this fits in at a national, international level was also discussed.

In these recessionary times, there is understandably an increasing pressure to prove your project's efficiency and effectiveness. The ERINA+ project have been working on developing tools for evaluating the scientific impact of EU-funded infrastructures. They shared their preliminary results with us this afternoon at the conference. Take it from me, tasked with measuring our own project's impact, it is a real challenge to capture all the predicted and unprecedented outcomes of our own dissemination project. It has been instructive learning about their methodology, and seeing the  ERINA+ project standardise their self-evaluation web tool for 21 projects.

So far, the aggregated (preliminary) results show that these projects have collectively conducted 213 dissemination activities reaching out to 246,000 persons in Europe and outside including United States, Latin America and China. Four spin-offs have resulted (no details provided in the slides). The projects have also been quite prolific! In total, 114 papers with impact factor have been published, and a further 62 papers without impact factor (plus 96 articles in conference proceedings). ERINA+ will provide a personalised report for projects later in the year, and will presenting their findings at eChallenges in October.

Watch our interview with Andrea Manieri (ERINA+ project coordinator) to find out more.

No comments: