{"id":19153773,"url":"https://github.com/christimperley/se-artifact-sharing","last_synced_at":"2026-06-16T12:32:19.600Z","repository":{"id":37214572,"uuid":"283565050","full_name":"ChrisTimperley/se-artifact-sharing","owner":"ChrisTimperley","description":"Provides the accompanying artifact for \"Understanding and Improving Artifact Sharing in Software Engineering Research\".","archived":false,"fork":false,"pushed_at":"2022-12-08T07:55:16.000Z","size":1871,"stargazers_count":0,"open_issues_count":10,"forks_count":0,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-02-22T21:22:47.844Z","etag":null,"topics":["artefact","artifact","replication-package","research","software-engineering"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ChrisTimperley.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-07-29T17:46:39.000Z","updated_at":"2021-05-04T16:02:41.000Z","dependencies_parsed_at":"2022-06-22T14:09:45.824Z","dependency_job_id":null,"html_url":"https://github.com/ChrisTimperley/se-artifact-sharing","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ChrisTimperley/se-artifact-sharing","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChrisTimperley%2Fse-artifact-sharing","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChrisTimperley%2Fse-artifact-sharing/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChrisTimperley%2Fse-artifact-sharing/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChrisTimperley%2Fse-artifact-sharing/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ChrisTimperley","download_url":"https://codeload.github.com/ChrisTimperley/se-artifact-sharing/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ChrisTimperley%2Fse-artifact-sharing/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34406820,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-16T02:00:06.860Z","response_time":126,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artefact","artifact","replication-package","research","software-engineering"],"created_at":"2024-11-09T08:24:13.255Z","updated_at":"2026-06-16T12:32:19.583Z","avatar_url":"https://github.com/ChrisTimperley.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Replication Package: Understanding and Improving Artifact Sharing in Software Engineering Research\n\nThis repository provides the associated publication analysis dataset and survey instrument\nfor the research paper,\n[Understanding and Improving Artifact Sharing in Software Engineering Research](http://dx.doi.org/10.1007/s10664-021-09973-5),\npublished in [Empirical Software Engineering](https://www.springer.com/journal/10664).\n(http://dx.doi.org/10.1007/s10664-021-09973-5).\nA preprint for the paper is available on arXiv: https://arxiv.org/abs/2008.01046.\n\nThis artifact is archived on Zenodo (https://doi.org/10.5281/zenodo.4737346) and hosted\non GitHub (https://github.com/ChrisTimperley/se-artifact-sharing).\n\nThe study was designed, conducted, and reported by the following investigators:\n\n* [Christopher S. Timperley](https://christimperley.co.uk) (Carnegie Mellon University)\n* [Lauren Herckis](http://www.laurenherckis.com) (Carnegie Mellon University)\n* [Claire Le Goues](https://clairelegoues.com) (Carnegie Mellon University)\n* [Michael Hilton](http://www.cs.cmu.edu/~mhilton) (Carnegie Mellon University)\n\nIf you have any questions about the research or this replication package, you should\ncontact Christopher or Michael.\n\n\n## Contents\n\nBelow is a description of the contents of this repository:\n\n* [`data/publication-analysis.yml`](data/publication-analysis.yml):\n  contains the results of our publication analysis.\n  A description of the format of this file is given below.\n* [`data/aec.csv`](data/aec.csv):\n  contains data on the number of artifacts submitted to and accepted by the\n  artifact evaluation committee at FSE between 2015 and 2018, inclusive.\n  We also include a count of the number of papers that we determined to contain\n  artifacts, based on the results of our publication analysis.\n* [`data/author-survey-quantitative.csv`](data/author-survey-quantitative.csv):\n  contains the quantitative results from our author survey.\n  A description of the format of this file is given below.\n* [`survey-questionnaire.pdf`](./survey-questionnaire.pdf):\n  outlines the questionnaire that was given to\n  participants and describes the branching logic that was used to target\n  specific subpopulations (e.g., past AEC members).\n* [`recruitment-email.pdf`](./recruitment-email.pdf):\n  provides the recruitment email that was sent to 744\n  validated participants, all of whom had published at least one technical-track\n  paper at ICSE, FSE, ASE, or EMSE in 2018.\n* [`consent-form.pdf`](./consent-form.pdf):\n  provides the form that was used by participants to\n  give their consent for the investigators to use their responses as part of the\n  study.\n* [`se-artifacts.ipynb`](./se-artifacts.ipynb):\n  provides the [Jupyter notebook](https://jupyter.org/) used to produce the\n  figures in the paper.\n* [`scripts/dblp.py`](scripts/dblp.py):\n  provides the script that was used to obtain a list of technical papers at\n  ICSE, FSE, ASE, and EMSE between 2014 and 2018, inclusive.\n* [`scripts/grab-urls.py`](scripts/grab-urls.py):\n  provides the script that was used to obtain a list of potential artifact\n  URLs from a given PDF.\n* [`data/dblp-2018-11-01.xml.gz`](data/dblp-2018-11-01.xml.gz):\n  the DBLP snapshot that was used to construct the publication dataset.\n* [`data/dois.txt`](data/dois.txt):\n  a newline-delimited list of the DOIs of the technical papers published at\n  ICSE, FSE, ASE, and EMSE between 2014 and 2018, inclusive.\n* [`paper.pdf`](./paper.pdf):\n  provides a preprint of the paper.\n\nNote that, due to the risk of reidentification, we do not provide the qualitative\nresults from our survey of authors as part of our replication package.\n\n\n### Publication Analysis\n\nThe publication analysis, represented as a YAML document, describes all technical\ntrack publications at [ICSE](http://www.icse-conferences.org/),\n[ASE](https://dl.acm.org/conference/ase),\n[FSE](https://dl.acm.org/conference/fse),\nand [EMSE](https://www.springer.com/journal/10664)\nbetween 2014 and 2018, inclusive.\nEach paper is described as an object with the following fields:\n\n* `authors`: a list of the authors of the paper, given by their `name` and\n  `affiliations` according to the paper.\n* `title`: the title of the paper.\n* `venue`: the venue at which the paper was published\n  (i.e., `ICSE`, `FSE`, `ASE`, or `EMSE`).\n* `pages`: the pages at which the paper appears in the associated journal\n  edition or conference proceedings, given as `start-end`, where `end`\n  is inclusive (e.g., `2900-2909`).\n* `year`: the year in which the paper was published.\n* `doi`: the DOI (digital object identifier) for the paper.\n* `artifact-links`: a list of the artifacts included in the paper that\n  are associated with that paper (i.e., we do not count artifacts produced\n  by another author/study). Each link is described by its `url`, a\n  boolean flag indicating whether or not the artifact was accessible upon\n  inspection `is-alive`, and, in the case that the URL redirects to another,\n  `redirects-to` gives the destination URL.\n\n**Provenance:**\nThe contents of the document were obtained through a combination of (a)\nscraping [dblp](https://dblp.uni-trier.de/), and (b) manually identifying and\nchecking the liveness of the links contained in each paper.\n\n\n### Author Survey Quantitative Responses\n\nWe provide responses to the quantitative components of our author survey in the form\nof a CSV file. Single choice questions are represented by a single column (e.g., `Q2`),\nwhereas multiple choice questions are given a separate column for each predefined\nresponse (e.g., `Q5_2`). The first row contains the number of the associated question,\nand the second row states the question itself.\nTo preserve anonymity, we removed participant numbers and shuffled responses.\n\n\n### Jupyter Notebook\n\nThe dependencies required to use our notebook can be installed using\n[pipenv](https://pypi.org/project/pipenv/), a popular tool for managing Python\nvirtual environments.\n\n```\n$ pipenv install\n```\n\nTo open our notebook using the classic Jupyter Notebook:\n\n```\n$ pipenv run jupyter notebook se-artifacts.ipynb\n```\n\nTo open our notebook using Jupyter Lab:\n\n```\n$ pipenv run jupyter-lab se-artifacts.ipynb\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchristimperley%2Fse-artifact-sharing","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchristimperley%2Fse-artifact-sharing","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchristimperley%2Fse-artifact-sharing/lists"}