{"id":271,"url":"https://github.com/dspinellis/awesome-msr","last_synced_at":"2025-10-24T16:30:49.981Z","repository":{"id":50556768,"uuid":"58886472","full_name":"dspinellis/awesome-msr","owner":"dspinellis","description":"A curated repository of software engineering repository mining data sets","archived":false,"fork":false,"pushed_at":"2021-01-02T20:42:55.000Z","size":93,"stargazers_count":404,"open_issues_count":0,"forks_count":67,"subscribers_count":36,"default_branch":"master","last_synced_at":"2024-05-20T04:41:00.527Z","etag":null,"topics":["awesome","awesome-list","dataset","ghtorrent","mining","msr"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"cc0-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dspinellis.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"contributing.md","funding":null,"license":"LICENSE","code_of_conduct":"code-of-conduct.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-05-15T21:57:18.000Z","updated_at":"2024-05-12T22:58:02.000Z","dependencies_parsed_at":"2022-09-19T15:20:13.554Z","dependency_job_id":null,"html_url":"https://github.com/dspinellis/awesome-msr","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dspinellis%2Fawesome-msr","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dspinellis%2Fawesome-msr/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dspinellis%2Fawesome-msr/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dspinellis%2Fawesome-msr/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dspinellis","download_url":"https://codeload.github.com/dspinellis/awesome-msr/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":237894311,"owners_count":19383168,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["awesome","awesome-list","dataset","ghtorrent","mining","msr"],"created_at":"2024-01-05T20:12:50.658Z","updated_at":"2025-10-24T16:30:49.974Z","avatar_url":"https://github.com/dspinellis.png","language":null,"funding_links":[],"categories":["Computer Science","Technical","计算机科学","Computer Networking and Security","Others","Uncategorized","Live Site:   [searchAwesome](https://search-awesome.vercel.app/)","Other Lists","Related Awesome Lists","링크","Themed Directories"],"sub_categories":["awesome-*","Uncategorized","TeX Lists","👥 Contributors","Updated this week"],"readme":"# Awesome Empirical Software Engineering [![Awesome](https://awesome.re/badge.svg)](https://awesome.re)\nA curated repository of data sets and tools that can be used for conducting evidence-based, data-driven research on software systems.\nThis research approach is often termed [experimental, or empirical software engineering](https://en.wikipedia.org/wiki/Experimental_software_engineering).\nMany of the data sets can also be useful in research using [search-based software engineering](https://en.wikipedia.org/wiki/Search-based_software_engineering) methods.\nThe repository is named after the [Mining Software Repositories (MSR)](https://www.msrconf.org/) conference series.\nFor examples of such work see the MSR conference's [Hall of Fame](http://2016.msrconf.org/#/hall-of-fame).\n\n\n- This list requires your input for its continuous improvement.\n  Read the [contribution guide](contributing.md) for instructions on how\n  you can contribute.\n  Alternatively, you can send me an [email](mailto:dds@aueb.gr)\n  if you find the process too cumbersome or confusing.\n- For more awesome lists, see [awesome](https://github.com/sindresorhus/awesome).\n\n## Contents\n- [Repositories](#repositories)\n- [Data Sets](#data-sets)\n- [Tools](#tools)\n- [Research Outlets](#research-outlets)\n\n## Repositories\n\n- [ESEUR](https://github.com/Derek-Jones/ESEUR-code-data) All data used in the openly available book [Evidence-based Software Engineering](http://www.knosof.co.uk/ESEUR/index.html)\n- [Directory of MSR Datasets](https://authecesofteng.github.io/directory-msr-datasets/)\n- [FLOSSmole](https://flossmole.org/collection_details) - Collaborative collection and analysis of free/libre/open source project data.\n- [PROMISE](http://promise.site.uottawa.ca/SERepository/datasets-page.html) - About 20 datasets related to software engineering research.\n- [SIR](http://sir.unl.edu/portal/index.php) - Software-artifact infrastructure repository; Java, C, C++, and C# software together with test suites and fault data.\n- [Zenodo](http://zenodo.org/) - Software data collections in CERN's open-access repository.\n  - [Software Engineering Artifacts Can Really Assist Future Tasks](http://zenodo.org/communities/seacraft)\n  - [Empirical Software Engineering](https://zenodo.org/communities/empirical-software-engineering/)\n  - [Mining Software Repositories](https://zenodo.org/communities/msr/)\n\n## Data Sets\n\n- [AndroidTimeMachine](https://androidtimemachine.github.io) - Graph-based dataset of commit history of 8,431 real-world Android apps.\n- [AndroZoo](https://androzoo.uni.lu/) - Collection of Android Applications.\n- [Bug Prediction Dataset](http://bug.inf.usi.ch/index.php) - Collection of models and metrics from Eclipse JDT Core, PDE UI, Equinox Framework, Lucene, Mylyn, and their histories.\n- [Code Reviews](http://kin-y.github.io/miningReviewRepo/) - Code reviews of OpenStack, LibreOffice, AOSP, Qt, Eclipse.\n- [CoREBench](http://www.comp.nus.edu.sg/%7Erelease/corebench/) - Collection of 70 realistically Complex Regression Errors that were systematically extracted from the repositories and bug reports of four open-source software projects: Make, Grep, Findutils, and Coreutils.\n- [Cryptocurrency GitHub Activity and Market Cap Dataset](https://rvantonder.github.io/CryptOSS/) - Activity such as commits, stars, prices, and market cap of over 200 cryptocurrency projects on GitHub over time. Raw, historic data is also [available](https://zenodo.org/record/2595588#.XRuzuBNKhSM).\n- [Defects4J](https://github.com/rjust/defects4j) - Collection of 395 reproducible bugs collected with the goal of advancing software testing research.\n- [Eclipse AERI stacktraces](http://download.eclipse.org/scava/datasets/aeri_stacktraces/aeri_stacktraces.html) - Collection of stacktraces of Exceptions encountered by users of the Eclipse IDE, as retrieved by the AERI reporting system.\n- [Enron Spreadsheets and Emails](https://figshare.com/articles/Enron_Spreadsheets_and_Emails/1221767) - All the spreadsheets and emails used in the paper 'Enron's Spreadsheets and Related Emails: A Dataset and Analysis'.\n- [Findbugs-maven](https://github.com/istlab/maven_bug_catalog) - Set of FindBugs reports for the Java projects of the [Maven repository](https://maven.apache.org).\n- [GHTorrent](http://ghtorrent.org/) - Scalable, queriable, offline mirror of data offered through the GitHub REST API.\n- [GitHub Bug Dataset](http://www.inf.u-szeged.hu/~ferenc/papers/GitHubBugDataSet/) - Bug Dataset of 15 Java open-source projects characterized by static source code metrics.\n- [GitHub on Google BigQuery](https://cloud.google.com/bigquery/public-data/github) - GitHub data accessible through Google's BigQuery platform.\n- [Grammar Zoo](http://slebok.github.io/zoo/) - Collection of grammars of DSLs and GPLs, some extracted from metamodels and document schemata.\n- [KaVE](http://www.kave.cc/datasets) - Developer tool interaction data.\n- [Linux Kernel 4.21 Call Graphs](https://zenodo.org/record/2652487#.XRnvomUzb0o) - The Linux Kernel 4.21 Call Graphs produced using [CScout](https://github.com/dspinellis/cscout/). \n- [Maven metrics](https://github.com/bkarak/data_msr2015) - Collection of software complexity \u0026 sizing metrics for the [Maven Repository](https://maven.apache.org).\n- [Maven Dependency Graph](https://zenodo.org/record/1489120) - Snapshot of the whole Maven Central taken on September 6, 2018, stored in a graph database.\n- [mzdata](https://github.com/jxshin/mzdata) - Multi-extract and multi-level dataset of Mozilla issue tracking history.\n- [npm-miner](https://github.com/AuthEceSoftEng/msr-2018-npm-miner) - The dataset contains the analysis results of 5 open source software quality tools eslint, escomplex, nsp, jsinspect and sonarjs for 2000 popular (in terms of stars and downloads) npm packages.\n- [OCL Expressions on GitHub](https://github.com/tue-mdse/ocl-dataset) - Data set of 9188 OCL expressions originating from 504 EMF meta-models in 245 systematically selected GitHub repositories.\n- [RepoReapers Data Set](https://reporeapers.github.io) - Data set containing a collection of _engineered software projects_ from GHTorrent.\n- [Software Heritage Graph Dataset](https://doi.org/10.5281/zenodo.2583978) - Graph of the development history and file metadata of \u003e80 million software projects from various forges (GitHub, Gitlab, Debian, PyPI, Google Code, etc) in a deduplicated and unified representation ([paper here](https://dl.acm.org/citation.cfm?id=3341907)).\n- [STAMINA](http://stamina.chefbe.net/download) - (STAte Machine INference Approaches) data are used to benchmark techniques for learning deterministic finite state machines (FSMs).\n- [Stack Exchange](https://archive.org/details/stackexchange) - Anonymized dump of all user-contributed content on the Stack Exchange network.\n- [TravisTorrent](http://travistorrent.testroots.org) - Provides free and easy-to-use Traivs CI build analyses.\n- [Ultimate Debian Database (UDD)](https://wiki.debian.org/UltimateDebianDatabase) - Data about various aspects of Debian (e.g. packages, bugs, mainteners) in the same SQL database.\n- [Unified Bug Dataset](http://www.inf.u-szeged.hu/~ferenc/papers/UnifiedBugDataSet/) - Static source code based datasets which includes the Bugcatchers Bug Dataset, the [Bug Prediction Dataset](http://bug.inf.usi.ch/index.php), the [Eclipse Bug Dataset](https://www.st.cs.uni-saarland.de/softevo/bug-data/eclipse/), the [GitHub Bug Dataset](http://www.inf.u-szeged.hu/~ferenc/papers/GitHubBugDataSet/), some datasets from the [PROMISE](http://promise.site.uottawa.ca/SERepository/datasets-page.html) repository.\n- [Unix history](https://github.com/dspinellis/unix-history-repo) - Git repository with 46 years of Unix history evolution.\n\n## Tools\n- [astminer](https://github.com/JetBrains-Research/astminer) - Library and tool for mining of path-based representations of code and other data derived from ASTs.\n- [Boa](http://boa.cs.iastate.edu/) - Domain-specific language and infrastructure that eases mining software repositories.\n- [buckwheat](https://github.com/JetBrains-Research/buckwheat) - Multi-language tokenizer for extracting identifiers from source code.\n- [ckjm](http://www.spinellis.gr/sw/ckjm/) - Chidamber and Kemerer Java Metrics.\n- [Coming](https://github.com/SpoonLabs/coming/) - A Java framework for analyzing code changes and mining instances of change patterns from Git repositories.\n- [CryptOSS](https://github.com/rvantonder/CryptOSS) - Mine GitHub activity and market cap data for cryptocurrency projects.\n- [DbDeo](https://github.com/tushartushar/DbDeo) - Extract embedded SQL statements and detect database schema smells.\n- [Designite](http://www.designite-tools.com) - Compute source code metrics and detect a variety of implementation, design, and architecture smells for C#.\n- [DesigniteJava](https://github.com/tushartushar/DesigniteJava) - Compute source code metrics and detect a variety of implementation and design smells for Java.\n- [Diggit](https://github.com/jrfaller/diggit) - Agile Ruby Tool to analyze Git repositories.\n- [GitEvo](https://github.com/andrehora/gitevo) - Code evolution analysis for Git repositories.\n- [GrimoireLab](http://grimoirelab.github.io/) - Free/Libre/Open Source tools for Software Development Analytics.\n- [MetricMiner](http://www.github.com/mauricioaniche/metricminer2) - Lean Java DSL to mine and extract data (e.g. commits, developers, modifications, diffs) from Git and SVN repositories.\n- [Maven-miner](https://github.com/diverse-project/maven-miner) - Java tools and infrastructure to resolve the whole Maven dependency graph, hosted in Maven Central, in the form of a [Neo4j](https://neo4j.com/) Graph.\n- [Perceval](https://github.com/chaoss/grimoirelab-perceval) - Fetch repository data from tens of back-ends.\n- [Puppeteer](https://github.com/tushartushar/Puppeteer) - Detect configuration smells in Puppet code.\n- [PyDriller](https://github.com/ishepard/pydriller) - Python Framework to analyse Git repositories.\n- [qmcalc](https://github.com/dspinellis/cqmetrics) - Calculate quality metrics from C source code.\n- [reaper](https://github.com/RepoReapers/reaper) - Python tool to compute a score for a repository from GHTorrent. The score quantifies the extent to which the project contained within the repository is _engineered_.\n- [RefactoringMiner](https://github.com/tsantalis/RefactoringMiner) - Library/API for detection of refactorings in changes of Java code.\n- [VulData7](https://github.com/electricalwind/data7) - Java framework enabling the automated collection of commits fixing vulnerabilities that are reported in NVD (links NVD with Git).\n\n## Research Outlets\n- Outlets exclusively devoted to empirical software engineering research\n  - [Empirical Software Engineering journal](https://link.springer.com/journal/10664)\n  - [MSR: Mining Software Repositories conference](https://www.msrconf.org/)\n  - [PROMISE: Predictive Models and Data Analytics in Software Engineering conference](http://promise.site.uottawa.ca/SERepository/)\n- Outlets that publish empirical software engineering research\n  - [ACM Transactions on Software Engineering and Methodology (TOSEM)](https://dl.acm.org/citation.cfm?id=J790)\n  - [ESEC/FSE: ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering](https://www.esec-fse.org/)\n  - [ICSE: International Conference on Software Engineering](http://www.icse-conferences.org/)\n  - [IEEE Software magazine](https://publications.computer.org/software-magazine/)\n  - [IEEE Transactions on Software Engineering](https://www.computer.org/csdl/journal/ts)\n  - [Journal of Systems and Software](https://www.journals.elsevier.com/journal-of-systems-and-software)\n  - [SANER: IEEE International Conference on Software Analysis, Evolution and Reengineering](https://ieeexplore.ieee.org/xpl/conhome.jsp?punumber=1000695)\n\n\n## License\n\n[![CC0](http://mirrors.creativecommons.org/presskit/buttons/88x31/svg/cc-zero.svg)](https://creativecommons.org/publicdomain/zero/1.0/)\n\nTo the extent possible under law, [Diomidis Spinellis](http://www.spinellis.gr) has waived all copyright and related or neighboring rights to this work.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdspinellis%2Fawesome-msr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdspinellis%2Fawesome-msr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdspinellis%2Fawesome-msr/lists"}