Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/speciesfilegroup/credit_digital_science

A thought experiment. A semi-currated list for administrators documenting how/why scientific researchers should get credit for their digital contributions.
https://github.com/speciesfilegroup/credit_digital_science

administration citation productivity science

Last synced: 3 months ago
JSON representation

A thought experiment. A semi-currated list for administrators documenting how/why scientific researchers should get credit for their digital contributions.

Awesome Lists containing this project

README 

          
# Credit digital science



A thought experiment. There must be somehwere administrators can go to find out how and why they should credit a researchers digital contributions, right?  We'll replace this with that place when we find it.



 A semi-currated list for administrators documenting how/why scientific researchers should get credit for their digital contributions.



# Contributing



Best: Fork the repository, make a pull request. Good: Submit an issue.



# Related links



* [Orcid IDs](https://orcid.org/)

* [Data carpentry](http://www.datacarpentry.org/)

* [Force11](https://www.force11.org/)



# Digital Productivity



## Products



Must meet each of the following:



* Citable

* Accessible beyond the individual researcher

* Published, public repository

* Published, private repository [use by invitation, sign-in]



And be in one of these categories:



## Digital assets

* Data compilations (e.g. databases, controlled vocabularies, ontologies, spreadsheets, digital image sets, shapefiles)

* Software (e.g. code libraries, desktop or mobile applications, scripts)

* Digital software services (e.g. application programming interfaces (APIs), software as a service layer, wrappers to computational resources)

* Technical documents (e.g. standards like ontologies, controlled vocabularies, schemas, software manuals, video tutorials, standard operating procedures)

* Technical illustrations (e.g. animations communicating scientific concepts, cover illustrations for journals)



## Services

* Hardware systems administration (e.g. managing a multi-user computational cluster, managing research-supporting clouds)

* Software development administration (e.g. defining programmers tasks, merging code, reviewing code, communicating with developers support channels)

* Technical software administration (e.g. administration of relational database, administrating research support wikis, administrating computing cluster software, administering scientific applications on the web or networks)

* Data manipulation (e.g. data extraction, data merging, data cleanup, data validation, data packaging, instantiating databases)

* Data archiving (e.g. properly registering datasets, annotating datasets with citable identifiers and appropriate standards, validating data in archives and archive derivatives)

* Informatics Education (e.g. teaching researchers software carpentry principles, database management, scripted data manipulation, principles of data archiving, principles of data standards)



## Metrics



Metrics are dependant on the type of product being evaluated.  It is critical to note that products in these categories have a diverse range of impacts, some of which can be far beyond the original intent.  The following are variously applicable, and not intended to represent a comprehensive set.  They should not be evaluated without a deeper understanding of what they represent.  For example a research stating that they made 10,000 changes to a code base may not be as important as a researcher who made 10 changes (i.e. they simply might have a different workflow, be not as effective, etc.).  Nevertheless when taken collectively each item here is important to evaluation.



* [Interesting take on managment](https://svpow.com/2017/03/17/every-attempt-to-manage-academia-makes-it-worse/)



### Digital asset metrics

* Number of times asset has been downloaded (e.g. downloading a software library)

* Number of times a digital asset has been referenced in a publically accessible format (e.g. references can be DOIs, but also things like URIs from ontologies or controlled vocabularies)

* Number of times asset has been accessed (e.g. page hits)

* Number of times an asset has been independently contributed to (e.g. outside researchers voluntarily contributing code to open source code libraries, this metric is particularly important)

* Number of dependencies (e.g. if a software library is produced, adopted by other external software developers, and required, then it is particularly impactful; determinable from sites like Github

* Number of open source libraries pushed to repositories

* Number of containerized applications pushed to repositories (containerized applications are applications that can be independently deployed to the cloud, their maintanence and packaging is an important technical contribution)

* Number of researchers commits to a code base

* Number of code bases committed to (e.g. a research committing small amounts to many applications or code bases will have a very big impact; one possible metric is the number of accepted “pull requests”, a standard mechanism for adding code to someone else’s project)

* Number of lines of code written (note this says nothing of quality of code)

* User time spent on digital application

* Number of data changes by OTHER users to digital resources managed by researcher (e.g. if I make a tool that 1000 researchers use to alter data this is immensely impactful, much more so than, say, a paper that gets cited 20 times)

* Number of users using a tool

* Number of logins of users using a tool

* Frequency of users returning to a tool

* Lines of code documented, lines of documentation written

* Hours of digital tutorials produced

* Number of project stars (e.g. on Github a star indicates someone else has more than a passing interest in a project)

* … many others



### Digital service metrics

* Number of external contributors “managed” (open source software development is highly impactful if external contributors are involved, this takes a lot of time to manage, but is highly rewarding)

* Number of code reviews done

* Lines of data extracted, manipulated, etc.

* Number of datasets manipulated

* Number of machines managed

* Number of databases managed

* Size of databases managed

* Longevity of databases managed (seems abstract, but is increasingly important)

* Types of software used to manage machine (e.g. address concept of technical competence)

* Types of software managed in support of science (e.g. managing relational databases is much more demanding than desktop applications installed once and forgotten)

* Number of researchers helped with digital problems (side by side)

* Number of issues closed (application, database, code support)

* Number of responses to ticketed issues (times digital support provided)

* … many others