{"id":20859371,"url":"https://github.com/engineeringsoftware/rustlang-coverage-study","last_synced_at":"2025-12-25T22:17:18.982Z","repository":{"id":250516339,"uuid":"834680742","full_name":"EngineeringSoftware/rustlang-coverage-study","owner":"EngineeringSoftware","description":"Compare performance of coverage tools for Rust and C++","archived":false,"fork":false,"pushed_at":"2024-07-28T11:14:17.000Z","size":12,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-19T07:26:13.054Z","etag":null,"topics":["coverage","cpp","rust"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/EngineeringSoftware.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-28T03:09:43.000Z","updated_at":"2024-07-30T18:07:50.000Z","dependencies_parsed_at":"2025-01-19T07:35:22.431Z","dependency_job_id":null,"html_url":"https://github.com/EngineeringSoftware/rustlang-coverage-study","commit_stats":null,"previous_names":["1sand0s/rustling-coverage-study"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EngineeringSoftware%2Frustlang-coverage-study","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EngineeringSoftware%2Frustlang-coverage-study/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EngineeringSoftware%2Frustlang-coverage-study/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EngineeringSoftware%2Frustlang-coverage-study/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/EngineeringSoftware","download_url":"https://codeload.github.com/EngineeringSoftware/rustlang-coverage-study/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243230095,"owners_count":20257644,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["coverage","cpp","rust"],"created_at":"2024-11-18T04:49:38.825Z","updated_at":"2025-12-25T22:17:13.904Z","avatar_url":"https://github.com/EngineeringSoftware.png","language":"Python","readme":"This repository contains a simple benchmark to compare the performance of\ncoverage analysis tools for Rust and C++.\n\n## Table of contents\n\n1. [Requirements](#Requirements)\n2. [Usage](#Usage)\n\n## Requirements\n\nThe packaged python script installs all the requirements except `gcov`, `poetry` and `python 3.12.x` which \nmust be installed manually and must be available on `Path`. The requirements are listed for\nreference.\n\n1. Rust language framework and tools.\n\n2. [GoogleTest](https://github.com/google/googletest). Test framework for C++\n\n3. [llvm-cov](https://github.com/taiki-e/cargo-llvm-cov) and [Tarpaulin](https://github.com/xd009642/tarpaulin). Coverage tools for Rust.\n\n4. [Gcov](https://gcc.gnu.org/onlinedocs/gcc/Gcov.html). Coverage tool for C++.\n\n5. [Poetry](https://python-poetry.org). Dependency management for Python.\n\n6. Python 3.12.x. Handles scripting.\n\n## Usage\n\n1. Use `poetry` to install script (`main.py`) dependencies within a virtual environment.\n   ```\n   poetry shell\n   poetry install\n   ```\n   \n2. Run script within virtual environment setup by poetry to collect coverage for C++ or Rust or both.\n   In the example below, we collect coverage for workloads with 100, 1000 and 10000 tests with each \n   test-suite execution time being averaged over `2` runs for C++ and Rust. \n   ```\n   python main.py --collect-coverage-overhead 100 1000 10000  --average 2\n   ```\n   \n3. Output for example given in `Step 2`\n   ```\n   Execution times in seconds for tests with coverage\n   ┌──────────────────┬─────────────────┬─────────────────┬─────────────────┐\n   │ #Tests           │ 100             │ 1000            │ 10000           │\n   ├──────────────────┼─────────────────┼─────────────────┼─────────────────┤\n   │ Rust (llvm-cov)  │ 0.4845 ± 0.0021 │ 1.2965 ± 0.0516 │ 10.611 ± 0.0608 │\n   │ Rust (tarpaulin) │ 0.689 ± 0.0198  │ 1.9045 ± 0.0078 │ 15.587 ± 0.0099 │\n   │ Cpp (gcov)       │ 0.002 ± 0.0     │ 0.008 ± 0.0     │ 0.0695 ± 0.0007 │\n   └──────────────────┴─────────────────┴─────────────────┴─────────────────┘\n   \n   \n   Execution times in seconds for tests without coverage\n   ┌──────────┬─────────────────┬─────────────────┬─────────────────┐\n   │ #Tests   │ 100             │ 1000            │ 10000           │\n   ├──────────┼─────────────────┼─────────────────┼─────────────────┤\n   │ Rust     │ 0.2625 ± 0.0191 │ 1.0335 ± 0.0035 │ 9.3525 ± 0.1025 │\n   │ Cpp      │ 0.001 ± 0.0     │ 0.006 ± 0.0     │ 0.055 ± 0.0     │\n   └──────────┴─────────────────┴─────────────────┴─────────────────┘\n   \n   \n   Coverage overhead as a ratio of execution time with and without coverage\n   ┌──────────────────┬───────┬────────┬─────────┐\n   │ #Tests           │   100 │   1000 │   10000 │\n   ├──────────────────┼───────┼────────┼─────────┤\n   │ Rust (llvm-cov)  │  1.85 │   1.25 │    1.13 │\n   │ Rust (tarpaulin) │  2.62 │   1.84 │    1.67 │\n   │ Cpp (gcov)       │  2    │   1.33 │    1.26 │\n   └──────────────────┴───────┴────────┴─────────┘ \n   ```\n\n4. For more script options\n   ```\n   python main.py --help\n   ```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fengineeringsoftware%2Frustlang-coverage-study","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fengineeringsoftware%2Frustlang-coverage-study","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fengineeringsoftware%2Frustlang-coverage-study/lists"}