{"id":44331570,"url":"https://github.com/mpecan/cargo-dupes","last_synced_at":"2026-02-17T20:01:19.442Z","repository":{"id":337770154,"uuid":"1154835201","full_name":"mpecan/cargo-dupes","owner":"mpecan","description":"A cargo subcommand that detects duplicate and near-duplicate code blocks in Rust codebases using AST normalization","archived":false,"fork":false,"pushed_at":"2026-02-11T09:50:38.000Z","size":153,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-11T14:29:08.722Z","etag":null,"topics":["ast","cargo","cargo-subcommand","code-quality","duplicate-detection","lint","rust","static-analysis"],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mpecan.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-10T20:38:44.000Z","updated_at":"2026-02-11T09:50:31.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/mpecan/cargo-dupes","commit_stats":null,"previous_names":["mpecan/cargo-dupes"],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/mpecan/cargo-dupes","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mpecan%2Fcargo-dupes","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mpecan%2Fcargo-dupes/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mpecan%2Fcargo-dupes/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mpecan%2Fcargo-dupes/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mpecan","download_url":"https://codeload.github.com/mpecan/cargo-dupes/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mpecan%2Fcargo-dupes/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29556406,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-17T18:16:07.221Z","status":"ssl_error","status_checked_at":"2026-02-17T18:16:04.782Z","response_time":100,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ast","cargo","cargo-subcommand","code-quality","duplicate-detection","lint","rust","static-analysis"],"created_at":"2026-02-11T10:06:46.143Z","updated_at":"2026-02-17T20:01:19.436Z","avatar_url":"https://github.com/mpecan.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# cargo-dupes\n\nA cargo subcommand that detects duplicate and near-duplicate code blocks in Rust codebases.\n\n## How It Works\n\n`cargo-dupes` parses Rust source files into ASTs using [syn](https://github.com/dtolnay/syn), then normalizes each function, method, and closure into a canonical form where:\n\n- **Identifiers are replaced** with positional placeholders (so `foo(x)` and `bar(y)` are identical)\n- **Literal values are erased** but types preserved (`42` and `99` are both \"integer literal\")\n- **Control flow structure is preserved** exactly\n- **Macro invocations become opaque** nodes\n\nThis normalized AST is hashed into a fingerprint for exact duplicate detection, and compared tree-by-tree using the Dice coefficient for near-duplicate detection.\n\n## Installation\n\n```sh\ncargo install --path .\n```\n\nOr, to run directly:\n\n```sh\ncargo run -- report\n```\n\nWhen installed, it's available as a cargo subcommand:\n\n```sh\ncargo dupes report\n```\n\n## Usage\n\n```\ncargo dupes [OPTIONS] [COMMAND]\n\nCommands:\n  stats    Show duplication statistics only\n  report   Show full duplication report (default)\n  check    Check for duplicates and exit with non-zero if thresholds exceeded\n  ignore   Add a fingerprint to the ignore list\n  ignored  List all ignored fingerprints\n\nOptions:\n  -p, --path \u003cPATH\u003e            Path to analyze (defaults to current directory)\n      --min-nodes \u003cMIN_NODES\u003e  Minimum AST node count for analysis [default: 10]\n      --min-lines \u003cMIN_LINES\u003e  Minimum source line count for analysis [default: 0 (disabled)]\n      --threshold \u003cTHRESHOLD\u003e  Similarity threshold for near-duplicates (0.0-1.0) [default: 0.8]\n      --format \u003cFORMAT\u003e        Output format [default: text] [possible values: text, json]\n      --exclude \u003cEXCLUDE\u003e      Exclude patterns (can be repeated)\n      --exclude-tests          Exclude test code (#[test] functions and #[cfg(test)] modules)\n```\n\n### Examples\n\n**Full report:**\n\n```sh\n$ cargo dupes report\nDuplication Statistics\n=====================\nTotal code units analyzed: 4\n\nExact duplicates: 1 groups (2 code units)\nNear duplicates:  0 groups (0 code units)\n\nDuplicated lines (exact): 18\nDuplicated lines (near):  0\nDuplication: 50.0% exact, 0.0% near (of 36 total lines)\n\nExact Duplicates\n================\n\nGroup 1 (fingerprint: 2a182da9e04e9428, 2 members):\n  - sum_positive (function) at src/lib.rs:2-10\n  - count_positive (function) at src/lib.rs:12-20\n```\n\n**Statistics only:**\n\n```sh\n$ cargo dupes stats\nDuplication Statistics\n=====================\nTotal code units analyzed: 4\n\nExact duplicates: 1 groups (2 code units)\nNear duplicates:  0 groups (0 code units)\n\nDuplicated lines (exact): 18\nDuplicated lines (near):  0\nDuplication: 50.0% exact, 0.0% near (of 36 total lines)\n```\n\n**JSON output:**\n\n```sh\n$ cargo dupes --format json stats\n{\n  \"total_code_units\": 4,\n  \"total_lines\": 36,\n  \"exact_duplicate_groups\": 1,\n  \"exact_duplicate_units\": 2,\n  \"near_duplicate_groups\": 0,\n  \"near_duplicate_units\": 0,\n  \"exact_duplicate_lines\": 18,\n  \"near_duplicate_lines\": 0,\n  \"exact_duplicate_percent\": 50.0,\n  \"near_duplicate_percent\": 0.0\n}\n```\n\n**CI check (fail if any exact duplicates exist):**\n\n```sh\n$ cargo dupes check --max-exact 0\n# Exits with code 1 if exact duplicate groups \u003e 0\n# Exits with code 0 if within thresholds\n```\n\n**CI check with percentage thresholds (fail if \u003e5% of lines are exact duplicates):**\n\n```sh\n$ cargo dupes check --max-exact-percent 5.0\n# Exits with code 1 if exact duplicate lines exceed 5% of total lines\n```\n\n**Exclude test code (inline `#[cfg(test)]` modules and `#[test]` functions):**\n\n```sh\n$ cargo dupes --exclude-tests report\n```\n\n**Exclude test directories by path:**\n\n```sh\n$ cargo dupes --exclude tests --exclude benches report\n```\n\n**Only report duplicates that are at least 10 lines long:**\n\n```sh\n$ cargo dupes --min-lines 10 report\n```\n\n**Lower the similarity threshold:**\n\n```sh\n$ cargo dupes --threshold 0.7 report\n```\n\n## Configuration\n\nConfiguration can be provided in three ways (in order of precedence):\n\n1. **CLI flags** (highest priority)\n2. **`dupes.toml`** in the project root\n3. **`Cargo.toml`** under `[package.metadata.dupes]`\n\n### `dupes.toml`\n\n```toml\nmin_nodes = 15\nmin_lines = 5\nsimilarity_threshold = 0.85\nexclude = [\"tests\", \"benches\"]\nexclude_tests = true\nmax_exact_duplicates = 0\nmax_near_duplicates = 10\nmax_exact_percent = 5.0\nmax_near_percent = 10.0\n```\n\n### `Cargo.toml`\n\n```toml\n[package.metadata.dupes]\nmin_nodes = 15\nsimilarity_threshold = 0.85\nexclude = [\"tests\"]\n```\n\n### Configuration Options\n\n| Option | Default | Description |\n|--------|---------|-------------|\n| `min_nodes` | `10` | Minimum AST node count for a code unit to be analyzed. Increase to skip trivial functions. |\n| `min_lines` | `0` | Minimum source line count for a code unit to be analyzed. `0` means disabled. |\n| `similarity_threshold` | `0.8` | Minimum similarity score (0.0-1.0) for near-duplicate detection. |\n| `exclude` | `[]` | Path patterns to exclude from scanning (substring match). |\n| `exclude_tests` | `false` | Exclude `#[test]` functions and `#[cfg(test)]` modules from analysis. |\n| `max_exact_duplicates` | `None` | For `check` subcommand: maximum allowed exact duplicate groups. |\n| `max_near_duplicates` | `None` | For `check` subcommand: maximum allowed near-duplicate groups. |\n| `max_exact_percent` | `None` | For `check` subcommand: maximum allowed exact duplicate line percentage. |\n| `max_near_percent` | `None` | For `check` subcommand: maximum allowed near-duplicate line percentage. |\n\n## Ignoring Duplicates\n\nSome duplicates are intentional (e.g., test helpers, trait implementations). You can ignore them by fingerprint:\n\n```sh\n# Add a fingerprint to the ignore list\n$ cargo dupes ignore 2a182da9e04e9428 --reason \"Intentional test helpers\"\nAdded 2a182da9e04e9428 to ignore list.\n\n# List ignored fingerprints\n$ cargo dupes ignored\nIgnored fingerprints:\n  2a182da9e04e9428 (reason: Intentional test helpers)\n\n# Ignored groups are automatically filtered from reports and checks\n$ cargo dupes report\n# The ignored group will not appear\n```\n\nThe ignore list is stored in `.dupes-ignore.toml` in the project root.\n\n## CI Integration\n\nUse the `check` subcommand in CI pipelines:\n\n```yaml\n# GitHub Actions example\n- name: Check for code duplication\n  run: cargo dupes check --max-exact 0 --max-exact-percent 5.0\n```\n\nExit codes:\n- **0** — Check passed (within thresholds)\n- **1** — Check failed (thresholds exceeded)\n- **2** — Error (no source files, invalid path, etc.)\n\n## What Gets Analyzed\n\n| Code Unit | Description |\n|-----------|-------------|\n| **Functions** | Top-level `fn` items |\n| **Methods** | `fn` items inside `impl` blocks |\n| **Trait impls** | `fn` items inside `impl Trait for Type` blocks |\n| **Closures** | Closure expressions (above the min node threshold) |\n\nThe scanner automatically:\n- Skips `target/` directories\n- Skips hidden directories (starting with `.`)\n- Respects exclude patterns\n- Handles parse errors gracefully (skips unparseable files with a warning)\n\n## Development\n\n**Requirements:** Rust 1.85+ (edition 2024)\n\n```sh\ncargo build          # Build\ncargo test           # Run all 147 tests\ncargo clippy         # Lint check\ncargo fmt --check    # Format check\n```\n\nPre-commit hooks (via `cargo-husky`) run clippy and rustfmt automatically.\n\n## License\n\nThis project is licensed under the [MIT License](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmpecan%2Fcargo-dupes","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmpecan%2Fcargo-dupes","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmpecan%2Fcargo-dupes/lists"}