{"id":13625851,"url":"https://github.com/jitsi/jiwer","last_synced_at":"2025-04-16T10:33:24.238Z","repository":{"id":39118719,"uuid":"137821392","full_name":"jitsi/jiwer","owner":"jitsi","description":"Evaluate your speech-to-text system with similarity measures such as word error rate (WER)","archived":false,"fork":false,"pushed_at":"2024-05-06T14:12:30.000Z","size":780,"stargazers_count":627,"open_issues_count":15,"forks_count":96,"subscribers_count":15,"default_branch":"master","last_synced_at":"2024-10-29T23:50:33.434Z","etag":null,"topics":["automatic-speech-recognition","evaluation-metrics","python3","speech-to-text","wer","word-error-rate"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jitsi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-06-19T00:38:38.000Z","updated_at":"2024-10-25T14:23:17.000Z","dependencies_parsed_at":"2023-02-16T14:45:21.676Z","dependency_job_id":"2b800433-05d9-44f8-aab8-49267b44930f","html_url":"https://github.com/jitsi/jiwer","commit_stats":{"total_commits":76,"total_committers":12,"mean_commits":6.333333333333333,"dds":0.618421052631579,"last_synced_commit":"372e0ea30dfbb506f6e271c0eac3e0b9035b0799"},"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jitsi%2Fjiwer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jitsi%2Fjiwer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jitsi%2Fjiwer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jitsi%2Fjiwer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jitsi","download_url":"https://codeload.github.com/jitsi/jiwer/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223708390,"owners_count":17189771,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automatic-speech-recognition","evaluation-metrics","python3","speech-to-text","wer","word-error-rate"],"created_at":"2024-08-01T21:02:03.769Z","updated_at":"2025-04-16T10:33:24.219Z","avatar_url":"https://github.com/jitsi.png","language":"Python","funding_links":[],"categories":["Python","Evaluation and Monitoring"],"sub_categories":[],"readme":"# JiWER\n\nJiWER is a simple and fast python package to evaluate an automatic speech recognition system.\nIt supports the following measures:\n\n1. word error rate (WER)\n2. match error rate (MER)\n3. word information lost (WIL) \n4. word information preserved (WIP) \n5. character error rate (CER)\n\nThese measures are computed with the use of the minimum-edit distance between one or more reference and hypothesis sentences.\nThe minimum-edit distance is calculated using [RapidFuzz](https://github.com/maxbachmann/RapidFuzz), which uses C++ under the hood, and is therefore faster than a pure python implementation.\n\n## Documentation\n\nFor further info, see the documentation at [jitsi.github.io/jiwer](https://jitsi.github.io/jiwer).\n\n## Installation\n\nYou should be able to install this package using [uv](https://docs.astral.sh/uv/): \n\n```\n$ uv add jiwer\n```\n\nOr, if you prefer old-fashioned pip and you're using Python \u003e= `3.8`:\n\n```bash\n$ pip install jiwer\n```\n\n## Usage\n\nThe most simple use-case is computing the word error rate between two strings:\n\n```python\nfrom jiwer import wer\n\nreference = \"hello world\"\nhypothesis = \"hello duck\"\n\nerror = wer(reference, hypothesis)\n```\n\n\n## A note on empty references\n\nThere is undefined behaviour when you apply an empty reference and hypothesis pair\nto the WER formula, as you get a division by zero.\n\nAs of version 4.0, `jiwer` defines the behaviour as follows, and thereby also\nlifts the requirement for reference strings to be non-empty.\nThis allows for testing whether models hallucinate on silent audio.\nNote that usually, there are multiple reference and hypothesis pairs.\nIt now supported that one or more of these references are empty, but to test well,\nmost references should still be non-empty.\n\n```python3\nimport jiwer\n\n# when ref and hyp are both empty, there is no error as\n# an ASR system correctly predicted silence/non-speech.\nassert jiwer.wer('', '') == 0 \nassert jiwer.mer('', '') == 0\nassert jiwer.wip('', '') == 1\nassert jiwer.wil('', '') == 0\n\nassert jiwer.cer('', '') == 0\n```\n\nWhen the hypothesis is non-empty, every word or character counts as an insertion:\n```python3\nimport jiwer\n\nassert jiwer.wer('', 'silence') == 1\nassert jiwer.wer('', 'peaceful silence') == 2\nassert jiwer.process_words('', 'now defined behaviour').insertions == 3\n\nassert jiwer.cer('', 'a') == 1\nassert jiwer.cer('', 'abcde') == 5\n```\n\n## Licence\n\nThe jiwer package is released under the `Apache License, Version 2.0` licence by [8x8](https://www.8x8.com/).\n\nFor further information, see [`LICENCE`](./LICENSE).\n\n## Reference\n\n_For a comparison between WER, MER and WIL, see: \\\nMorris, Andrew \u0026 Maier, Viktoria \u0026 Green, Phil. (2004). [From WER and RIL to MER and WIL: improved evaluation measures for connected speech recognition.](https://www.researchgate.net/publication/221478089_From_WER_and_RIL_to_MER_and_WIL_improved_evaluation_measures_for_connected_speech_recognition)_\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjitsi%2Fjiwer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjitsi%2Fjiwer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjitsi%2Fjiwer/lists"}