awesome-msr
A curated repository of software engineering repository mining data sets
https://github.com/dspinellis/awesome-msr
Last synced: 23 days ago
JSON representation
Data Sets
- AndroZoo - Collection of Android Applications.
- Eclipse AERI stacktraces - Collection of stacktraces of Exceptions encountered by users of the Eclipse IDE, as retrieved by the AERI reporting system.
- STAMINA - (STAte Machine INference Approaches) data are used to benchmark techniques for learning deterministic finite state machines (FSMs).
- Stack Exchange - Anonymized dump of all user-contributed content on the Stack Exchange network.
- Ultimate Debian Database (UDD) - Data about various aspects of Debian (e.g. packages, bugs, mainteners) in the same SQL database.
- TravisTorrent - Provides free and easy-to-use Traivs CI build analyses.
- GHTorrent - Scalable, queriable, offline mirror of data offered through the GitHub REST API.
- Cryptocurrency GitHub Activity and Market Cap Dataset - Activity such as commits, stars, prices, and market cap of over 200 cryptocurrency projects on GitHub over time. Raw, historic data is also available.
- Defects4J - Collection of 395 reproducible bugs collected with the goal of advancing software testing research.
- Findbugs-maven - Set of FindBugs reports for the Java projects of the Maven repository.
- GitHub Bug Dataset - Bug Dataset of 15 Java open-source projects characterized by static source code metrics.
- Maven metrics - Collection of software complexity & sizing metrics for the Maven Repository.
- mzdata - Multi-extract and multi-level dataset of Mozilla issue tracking history.
- npm-miner - The dataset contains the analysis results of 5 open source software quality tools eslint, escomplex, nsp, jsinspect and sonarjs for 2000 popular (in terms of stars and downloads) npm packages.
- OCL Expressions on GitHub - Data set of 9188 OCL expressions originating from 504 EMF meta-models in 245 systematically selected GitHub repositories.
- Unified Bug Dataset - Static source code based datasets which includes the Bugcatchers Bug Dataset, the Bug Prediction Dataset, the Eclipse Bug Dataset, the GitHub Bug Dataset, some datasets from the PROMISE repository.
- Unix history - Git repository with 46 years of Unix history evolution.
- Code Reviews - Code reviews of OpenStack, LibreOffice, AOSP, Qt, Eclipse.
- Grammar Zoo - Collection of grammars of DSLs and GPLs, some extracted from metamodels and document schemata.
- Enron Spreadsheets and Emails - All the spreadsheets and emails used in the paper 'Enron's Spreadsheets and Related Emails: A Dataset and Analysis'.
- Linux Kernel 4.21 Call Graphs - The Linux Kernel 4.21 Call Graphs produced using CScout.
- Maven Dependency Graph - Snapshot of the whole Maven Central taken on September 6, 2018, stored in a graph database.
- Software Heritage Graph Dataset - Graph of the development history and file metadata of >80 million software projects from various forges (GitHub, Gitlab, Debian, PyPI, Google Code, etc) in a deduplicated and unified representation (paper here).
- AndroidTimeMachine - Graph-based dataset of commit history of 8,431 real-world Android apps.
- RepoReapers Data Set - Data set containing a collection of engineered software projects from GHTorrent.
- SWE-bench - SWE-bench is a benchmark designed to evaluate the ability of AI models to solve real-world software engineering problems by generating fixes for issues found in open-source code repositories.
Repositories
- ESEUR - based Software Engineering](http://www.knosof.co.uk/ESEUR/index.html)
- Directory of MSR Datasets
- FLOSSmole - Collaborative collection and analysis of free/libre/open source project data.
- PROMISE - About 20 datasets related to software engineering research.
- SIR - Software-artifact infrastructure repository; Java, C, C++, and C# software together with test suites and fault data.
- Zenodo - Software data collections in CERN's open-access repository.
- Software Engineering Artifacts Can Really Assist Future Tasks
- Empirical Software Engineering
- Mining Software Repositories
Research Outlets
- MSR: Mining Software Repositories conference
- ESEC/FSE: ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering
- IEEE Transactions on Software Engineering
- SANER: IEEE International Conference on Software Analysis, Evolution and Reengineering
- PROMISE: Predictive Models and Data Analytics in Software Engineering conference
- ACM Transactions on Software Engineering and Methodology (TOSEM)
- ICSE: International Conference on Software Engineering
- Journal of Systems and Software
- Empirical Software Engineering journal
Tools
- astminer - Library and tool for mining of path-based representations of code and other data derived from ASTs.
- buckwheat - Multi-language tokenizer for extracting identifiers from source code.
- CryptOSS - Mine GitHub activity and market cap data for cryptocurrency projects.
- DbDeo - Extract embedded SQL statements and detect database schema smells.
- DesigniteJava - Compute source code metrics and detect a variety of implementation and design smells for Java.
- Diggit - Agile Ruby Tool to analyze Git repositories.
- GitEvo - Code evolution analysis for Git repositories.
- GrimoireLab - Free/Libre/Open Source tools for Software Development Analytics.
- Maven-miner - Java tools and infrastructure to resolve the whole Maven dependency graph, hosted in Maven Central, in the form of a Neo4j Graph.
- Perceval - Fetch repository data from tens of back-ends.
- Puppeteer - Detect configuration smells in Puppet code.
- PyDriller - Python Framework to analyse Git repositories.
- qmcalc - Calculate quality metrics from C source code.
- reaper - Python tool to compute a score for a repository from GHTorrent. The score quantifies the extent to which the project contained within the repository is engineered.
- RefactoringMiner - Library/API for detection of refactorings in changes of Java code.
- VulData7 - Java framework enabling the automated collection of commits fixing vulnerabilities that are reported in NVD (links NVD with Git).
- Coming - A Java framework for analyzing code changes and mining instances of change patterns from Git repositories.
- MetricMiner - Lean Java DSL to mine and extract data (e.g. commits, developers, modifications, diffs) from Git and SVN repositories.
- Boa - Domain-specific language and infrastructure that eases mining software repositories.
- Designite - Compute source code metrics and detect a variety of implementation, design, and architecture smells for C#.