{"id":16871824,"url":"https://github.com/desh2608/spyder","last_synced_at":"2025-08-02T00:34:36.988Z","repository":{"id":43319918,"uuid":"340756841","full_name":"desh2608/spyder","owner":"desh2608","description":"Simple Python package for fast DER computation","archived":false,"fork":false,"pushed_at":"2023-06-29T19:54:48.000Z","size":101,"stargazers_count":33,"open_issues_count":1,"forks_count":7,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-06-08T21:44:11.383Z","etag":null,"topics":["der","diarization"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/desh2608.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2021-02-20T21:26:40.000Z","updated_at":"2025-03-21T12:59:17.000Z","dependencies_parsed_at":"2024-01-13T22:25:06.655Z","dependency_job_id":"a176ae5b-1af0-4879-b1bb-6871945a1776","html_url":"https://github.com/desh2608/spyder","commit_stats":{"total_commits":38,"total_committers":3,"mean_commits":"12.666666666666666","dds":"0.052631578947368474","last_synced_commit":"827de9e2be23e63948aac7078a39e1bdd3fef5fb"},"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/desh2608/spyder","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/desh2608%2Fspyder","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/desh2608%2Fspyder/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/desh2608%2Fspyder/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/desh2608%2Fspyder/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/desh2608","download_url":"https://codeload.github.com/desh2608/spyder/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/desh2608%2Fspyder/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265327786,"owners_count":23747775,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["der","diarization"],"created_at":"2024-10-13T15:09:51.136Z","updated_at":"2025-07-14T18:08:21.333Z","avatar_url":"https://github.com/desh2608.png","language":"C++","funding_links":[],"categories":["Software"],"sub_categories":["Evaluation"],"readme":"\u003ch1 align=\"center\"\u003eSPYDER\u003c/h1\u003e\n\nA simple Python package for fast DER computation.\n\n## Installation\n\n```shell\npip install spy-der\n```\n\nTo install version with latest features directly from Github:\n\n```shell\npip install git+https://github.com/desh2608/spyder.git@main\n```\n\nFor development, clone this repository and run:\n\n```shell\npip install --editable .\n```\n\n## Usage\n### Compute DER for a single pair of reference and hypothesis\n\n```python\nimport spyder\n\n# reference (ground truth)\nref = [(\"A\", 0.0, 2.0), # (speaker, start, end)\n       (\"B\", 1.5, 3.5),\n       (\"A\", 4.0, 5.1)]\n\n# hypothesis (diarization result from your algorithm)\nhyp = [(\"1\", 0.0, 0.8),\n       (\"2\", 0.6, 2.3),\n       (\"3\", 2.1, 3.9),\n       (\"1\", 3.8, 5.2)]\n\n# compute DER on full recording\nprint(spyder.DER(ref, hyp))\n# DERMetrics(duration=5.10,miss=9.80%,falarm=21.57%,conf=25.49%,der=56.86%)\n\n# compute DER on single-speaker regions only\nprint(spyder.DER(ref, hyp, regions=\"single\"))\n# DERMetrics(duration=4.10,miss=0.00%,falarm=26.83%,conf=19.51%,der=46.34%)\n\n# compute DER using UEM segments\nuem = [(0.5, 5.0)]\nprint(spyder.DER(ref, hyp, uem=uem))\n# DERMetrics(duration=4.50,miss=11.11%,falarm=22.22%,conf=26.67%,der=60.00%)\n\n# compute DER using collar\nprint(spyder.DER(ref, hyp, collar=0.2))\n# DERMetrics(duration=3.10,miss=3.23%,falarm=12.90%,conf=19.35%,der=35.48%)\n\n# get speaker mapping between reference and hypothesis\nmetrics = spyder.DER(ref, hyp)\nprint(f\"Reference speaker map: {metrics.ref_map}\")\nprint(f\"Hypothesis speaker map: {metrics.hyp_map}\")\n# Reference speaker map: {'A': '0', 'B': '1'}\n# Hypothesis speaker map: {'1': '0', '2': '2', '3': '1'}\n```\n\n### Compute DER for multiple pairs of reference and hypothesis\n\n```python\nimport spyder\n\n# for multiple pairs, reference and hypothesis should be lists or dicts\n# if lists, ref and hyp must have same length\n\n# reference (ground truth)\nref = {\"uttr0\":[(\"A\", 0.0, 2.0), # (speaker, start, end)\n                (\"B\", 1.5, 3.5),\n                (\"A\", 4.0, 5.1)],\n       \"uttr2\":[(\"A\", 0.0, 4.3), # (speaker, start, end)\n                (\"C\", 6.0, 8.1),\n                (\"B\", 2.0, 8.5)]}\n\n# hypothesis (diarization result from your algorithm)\nhyp = {\"uttr0\":[(\"1\", 0.0, 0.8),\n                (\"2\", 0.6, 2.3),\n                (\"3\", 2.1, 3.9),\n                (\"1\", 3.8, 5.2)],\n       \"uttr2\":[(\"1\", 0.0, 4.5),\n                (\"2\", 2.5, 8.7)]}\n\nmetrics = spyder.DER(ref, hyp)\nprint(metrics)\n# {'Overall': DERMetrics(duration=18.00,miss=17.22%,falarm=8.33%,conf=7.22%,der=32.78%)}\n\nmetrics2 = spyder.DER(ref, hyp, per_file=True, verbose=True)  # verbose=True to prints per-file results\n```\nOutput:\n```\nEvaluated 2 recordings on `all` regions. Results:\n╒═════════════╤════════════════╤═════════╤════════════╤═════════╤════════╕\n│ Recording   │   Duration (s) │   Miss. │   F.Alarm. │   Conf. │    DER │\n╞═════════════╪════════════════╪═════════╪════════════╪═════════╪════════╡\n│ uttr0       │           5.10 │   9.80% │     21.57% │  25.49% │ 56.86% │\n├─────────────┼────────────────┼─────────┼────────────┼─────────┼────────┤\n│ uttr2       │          12.90 │  20.16% │      3.10% │   0.00% │ 23.26% │\n├─────────────┼────────────────┼─────────┼────────────┼─────────┼────────┤\n│ Overall     │          18.00 │  17.22% │      8.33% │   7.22% │ 32.78% │\n╘═════════════╧════════════════╧═════════╧════════════╧═════════╧════════╛\n```\n\nAdditionally, you can provide UEM and collar parameters similar to single pair case.\n\n### Compute per-file and overall DERs between reference and hypothesis RTTMs using command line tool\n\nAlternatively, __spyder__ can also be invoked from the command line to compute the per-file\nand average DERs between reference and hypothesis RTTMs.\n\n```shell\nUsage: spyder [OPTIONS] REF_RTTM HYP_RTTM\n\nOptions:\n  -u, --uem PATH                  UEM file (format: \u003crecording_id\u003e \u003cchannel\u003e\n                                  \u003cstart\u003e \u003cend\u003e)\n\n  -p, --per-file                  If this flag is set, print per file results.\n                                  [default: False]\n\n  -s, --skip-missing              Skip recordings which are missing in\n                                  hypothesis (i.e., not counted in missed\n                                  speech).  [default: False]\n\n  -r, --regions [all|single|overlap|nonoverlap]\n                                  Only evaluate on the selected region type.\n                                  Default is all.  - all: all regions.  -\n                                  single: only single-speaker regions (ignore\n                                  silence and multiple speaker).  - overlap:\n                                  only regions with multiple speakers in the\n                                  reference.  - nonoverlap: only regions\n                                  without multiple speakers in the reference.\n                                  [default: all]\n\n  -c, --collar FLOAT RANGE        Collar size.  [default: 0.0]\n  -m, --print-speaker-map         Print speaker mapping for reference and\n                                  hypothesis speakers.  [default: False]\n\n  --help                          Show this message and exit.\n```\n\nExamples:\n\n```shell\n\u003e spyder ref_rttm hyp_rttm\nEvaluated 16 recordings on `all` regions. Results:\n╒═════════════╤════════════════╤═════════╤════════════╤═════════╤════════╕\n│ Recording   │   Duration (s) │   Miss. │   F.Alarm. │   Conf. │    DER │\n╞═════════════╪════════════════╪═════════╪════════════╪═════════╪════════╡\n│ Overall     │       33952.95 │  11.48% │      2.27% │   9.81% │ 23.56% │\n╘═════════════╧════════════════╧═════════╧════════════╧═════════╧════════╛\n\n\u003e spyder ref_rttm hyp_rttm -r single -p -c 0.25\nEvaluated 16 recordings on `single` regions. Results:\n╒═════════════════════╤════════════════╤═════════╤════════════╤═════════╤════════╕\n│ Recording           │   Duration (s) │   Miss. │   F.Alarm. │   Conf. │    DER │\n╞═════════════════════╪════════════════╪═════════╪════════════╪═════════╪════════╡\n│ EN2002a.Mix-Headset │        1032.05 │   0.00% │      2.98% │   4.97% │  7.94% │\n├─────────────────────┼────────────────┼─────────┼────────────┼─────────┼────────┤\n│ EN2002b.Mix-Headset │         853.56 │   0.00% │      3.40% │   5.39% │  8.80% │\n├─────────────────────┼────────────────┼─────────┼────────────┼─────────┼────────┤\n│ EN2002c.Mix-Headset │        1641.68 │   0.00% │      1.42% │   1.05% │  2.47% │\n├─────────────────────┼────────────────┼─────────┼────────────┼─────────┼────────┤\n│ EN2002d.Mix-Headset │        1006.27 │   0.00% │      3.12% │   7.14% │ 10.26% │\n├─────────────────────┼────────────────┼─────────┼────────────┼─────────┼────────┤\n│ ES2004a.Mix-Headset │         539.48 │   0.00% │      1.62% │   5.12% │  6.74% │\n├─────────────────────┼────────────────┼─────────┼────────────┼─────────┼────────┤\n│ ES2004b.Mix-Headset │        1582.05 │   0.00% │      0.82% │   1.39% │  2.21% │\n├─────────────────────┼────────────────┼─────────┼────────────┼─────────┼────────┤\n│ ES2004c.Mix-Headset │        1526.84 │   0.00% │      0.45% │   1.27% │  1.72% │\n├─────────────────────┼────────────────┼─────────┼────────────┼─────────┼────────┤\n│ ES2004d.Mix-Headset │        1172.72 │   0.00% │      1.77% │   9.60% │ 11.37% │\n├─────────────────────┼────────────────┼─────────┼────────────┼─────────┼────────┤\n│ IS1009a.Mix-Headset │         425.51 │   0.00% │      3.94% │   4.60% │  8.54% │\n├─────────────────────┼────────────────┼─────────┼────────────┼─────────┼────────┤\n│ IS1009b.Mix-Headset │        1412.03 │   0.00% │      1.23% │   0.85% │  2.08% │\n├─────────────────────┼────────────────┼─────────┼────────────┼─────────┼────────┤\n│ IS1009c.Mix-Headset │        1283.21 │   0.00% │      2.74% │   1.00% │  3.75% │\n├─────────────────────┼────────────────┼─────────┼────────────┼─────────┼────────┤\n│ IS1009d.Mix-Headset │        1164.49 │   0.00% │      2.27% │   3.37% │  5.64% │\n├─────────────────────┼────────────────┼─────────┼────────────┼─────────┼────────┤\n│ TS3003a.Mix-Headset │         804.27 │   0.00% │      0.00% │  11.28% │ 11.28% │\n├─────────────────────┼────────────────┼─────────┼────────────┼─────────┼────────┤\n│ TS3003b.Mix-Headset │        1509.49 │   0.00% │      0.36% │   0.75% │  1.11% │\n├─────────────────────┼────────────────┼─────────┼────────────┼─────────┼────────┤\n│ TS3003c.Mix-Headset │        1566.84 │   0.00% │      1.76% │   1.74% │  3.50% │\n├─────────────────────┼────────────────┼─────────┼────────────┼─────────┼────────┤\n│ TS3003d.Mix-Headset │        1357.45 │   0.00% │      2.42% │   2.93% │  5.35% │\n├─────────────────────┼────────────────┼─────────┼────────────┼─────────┼────────┤\n│ Overall             │       18877.94 │   0.00% │      1.72% │   3.29% │  5.01% │\n╘═════════════════════╧════════════════╧═════════╧════════════╧═════════╧════════╛\n```\n\n## Why spyder?\n\n* __Fast:__ Implemented in pure C++, and faster than the alternatives (md-eval.pl,\ndscore, pyannote.metrics). See this [benchmark](https://desh2608.github.io/2021-03-05-spyder/)\nfor comparisons with other tools.\n* __Stand-alone:__ It has no dependency on any other library. We have our own\nimplementation of the Hungarian algorithm, for example, instead of using `scipy`.\n* __Easy-to-use:__ No need to write the reference and hypothesis turns to files and\nread md-eval output with complex regex patterns.\n* __Overlap:__ Spyder supports overlapping speech in reference and hypothesis. In addition,\nyou can compute metrics on just the single-speaker or overlap regions by passing the\nkeyword argument `regions=\"single\"` or `regions=\"overlap\"`, respectively.\n\n\n## Contributing\n\nContributions for core improvements or new recipes are welcome. Please run the following\nbefore creating a pull request.\n\n```bash\npre-commit install\npre-commit run # Running linter checks\n```\n\n\n## Bugs/issues\n\nPlease raise an issue in the [issue tracker](https://github.com/desh2608/spyder/issues).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdesh2608%2Fspyder","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdesh2608%2Fspyder","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdesh2608%2Fspyder/lists"}