e-FISCAL, e.inventory and ERINA+ (read more about this in a recent iSGTW article).
|Daniel Katz, NSF|
So how do developers/architects of software and e-infrastructures know when their work is successful. And more importantly how does the outside world (especially funders) recognise exactly who collaborated and contributed to each and every scientific output. For the developer of an open source physics simulation possible metrics could include recording downloads and users as well as tracking paper citations. However, Katz suggests that what could be more valuable but harder to monitor are the number of citations each paper that cited your software receives. He emphasized the need to develop a system where credit is given to those involved in building the software and data sets. NSF have proposed a number of solutions -a vision for citations and metrics.
One recommendation is that all products - including software, data sets and workflows, are registered. Next, a credit map is developed. The input would be the weighted list of contributors (e.g. both people and products etc.) with the DOI as the final output. This could lead to transitive credit enabling developers to gather evidence for their applications widespread usability. Product usage could also be automatically recorded by simply incorporating a general usage code into each software package.
Commercial tools are generally tracked by the money they generate but this doesn't help academics understand what tools were used for what outcomes, says Katz. Tokens could be provided as part of science grants. Users could then distribute tokens either during or after using the tools.
Unfortunately, as researchers don't always acknowledge the full array of tools that helped them with their work developing an effective reward system for those developers and architects working behind the scenes will be vitally important going forward.