{"id":48914969,"url":"https://github.com/sjswerdloff/transcriber-radrx","last_synced_at":"2026-04-17T01:32:17.684Z","repository":{"id":350870832,"uuid":"1206685210","full_name":"sjswerdloff/transcriber-radrx","owner":"sjswerdloff","description":"A proposed framework for validating transcription, in particular for the radiation therapy domain","archived":false,"fork":false,"pushed_at":"2026-04-12T13:59:39.000Z","size":690,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-12T15:29:52.988Z","etag":null,"topics":["asr","locally-hosted","radiation-therapy","validation-framework"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sjswerdloff.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":"ROADMAP.md","authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":"NOTICE","maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-10T06:43:35.000Z","updated_at":"2026-04-12T13:59:43.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/sjswerdloff/transcriber-radrx","commit_stats":null,"previous_names":["sjswerdloff/transcriber-radrx"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/sjswerdloff/transcriber-radrx","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sjswerdloff%2Ftranscriber-radrx","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sjswerdloff%2Ftranscriber-radrx/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sjswerdloff%2Ftranscriber-radrx/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sjswerdloff%2Ftranscriber-radrx/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sjswerdloff","download_url":"https://codeload.github.com/sjswerdloff/transcriber-radrx/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sjswerdloff%2Ftranscriber-radrx/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31911478,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-16T18:22:33.417Z","status":"ssl_error","status_checked_at":"2026-04-16T18:21:47.142Z","response_time":69,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["asr","locally-hosted","radiation-therapy","validation-framework"],"created_at":"2026-04-17T01:32:16.640Z","updated_at":"2026-04-17T01:32:17.657Z","avatar_url":"https://github.com/sjswerdloff.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# transcriber-radrx\n\n[![CI](https://github.com/sjswerdloff/transcriber-radrx/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/sjswerdloff/transcriber-radrx/actions/workflows/ci.yml)\n\n**A validation framework and complementary-ensemble approach for clinical\nautomatic speech recognition (ASR) in radiation oncology dictation.**\n\nThis project provides two things:\n\n1. A **validation framework** for rigorously evaluating whether any ASR\n   backend is safe for clinical dictation — with metrics that go beyond\n   word error rate to catch the safety-critical failures that aggregate\n   statistics hide.\n2. A **2-backend ensemble** (Voxtral Mini 3B + Whisper large-v3) that\n   exploits complementary failure profiles between an audio-LLM and a\n   classical ASR to achieve **98.5% automatic resolution** with **zero\n   unrecoverable safety failures** across 72 radiation oncology fixtures,\n   leaving only **1.52% of words** for human review.\n\nWe don't have all the answers. We have a framework for asking the questions,\na growing collection of reproducible experiments, and a concrete result\nthat turns two individually unsafe ASR backends into an ensemble that is\ndemonstrably safer than either alone.\n\n---\n\n## What this project is\n\n- A **modular pipeline** for generating synthetic clinical speech, injecting\n  realistic acoustic and noise conditions, running it through multiple ASR\n  backends, and scoring the results with metrics that go beyond word error\n  rate (WER) to include clinical-vocabulary preservation and safety-critical\n  token audit.\n- A **multi-backend bake-off harness** that currently supports six ASR\n  backends (Whisper, MedASR, Cohere Transcribe, Granite-Speech 2B and 8B,\n  Voxtral Mini 3B) and is trivially extensible to any new backend that\n  implements a small Protocol interface.\n- A **safety-gate metric** with five clinically-derived failure classes\n  (decimal drop, dose value missing, silent unit substitution, dose unit\n  corruption, slashed form loss), each tagged with a correctability rating\n  (UNRECOVERABLE, CONTEXT_RULE, PHONETIC_MAP, ADJACENCY_RULE) and severity\n  weight calibrated by a radiation oncology physicist. Produces a formal\n  deployment gate: PASS / CONDITIONAL / FAIL — both for raw ASR output\n  and for predicted post-correction output.\n- A **2-backend ensemble** that aligns Voxtral and Whisper word-by-word,\n  applies 10 prioritized token-class decision rules at disagreement points,\n  and produces a single output with per-word provenance. Introduces **UWR**\n  (Unresolved Word Rate) — the fraction of words the ensemble cannot\n  confidently resolve and must defer to a downstream reviewer.\n- A **72-fixture corpus** across three domains: 24 dense clinical RT\n  fixtures, 28 particle therapy fixtures (proton + carbon ion, including\n  pediatric CSI with junctioned PBS), and 20 anatomy-coverage fixtures\n  (breast, prostate, cervix, rectum, anus, vulva, vagina, endometrium,\n  testes, penis, head \u0026 neck, lung).\n- A **Word .docx renderer** with Track Changes that produces two documents\n  from the same ensemble output: an *audit* document (every ensemble\n  decision visible as a tracked change for regulatory traceability) and a\n  *review* document (automated fixes baked in, only UWR items shown as\n  highlighted words with margin comments listing both ASR options).\n- A **reproducible noise-injection stage** using the MUSAN corpus with a\n  prefer-long-first splice strategy so that every transcription is covered\n  by continuous ambient noise at a known signal-to-noise ratio.\n- A **growing set of cycle reports** that document what we tested, what we\n  found, what flipped when we expanded the test scope, and what we decided\n  not to claim yet. The reports are the receipts.\n\n## What this project is not\n\n- It is **not a clinical product.** Nothing in this repository is certified,\n  validated, or approved for clinical use. Any deployment would require\n  independent validation against the specific clinical environment, voice\n  distribution, and vocabulary in use.\n- It is **not a leaderboard.** We are less interested in \"which ASR wins\"\n  than in \"how do you know you can trust the winner.\" The ranking is a\n  by-product of the process working.\n- It is **not a finished framework.** The cycle reports document work in\n  progress. Every cycle surfaces things the previous cycles got wrong or\n  under-sampled. The `ROADMAP.md` file is a living document of the open\n  work.\n\n## Why this matters\n\nClinical dictation is one of the few contexts where an ASR error can\ndirectly harm a patient. A silent decimal-point drop (`50.4 Gy` transcribed\nas `504 gy`) propagates a ten-times-lethal dose. A misread drug name\nsubstitutes one treatment for another. A misheard anatomy word changes\nthe target of a radiation field. These are not hypothetical — cycle 110\nof this project found a real instance of the decimal-drop failure in one\nof the models tested, hidden underneath a headline word-error-rate of\n9.25 % on the single voice where it happened.\n\nWord error rate alone will not catch these failures. The aggregate metric\noptimises over the median token; the dangerous ones are in the tails.\nA validation framework for clinical ASR has to look at the individual\nsafety-critical tokens, not just the average.\n\nThe framing is: *we don't have to have the answer, we just need a way of\nthinking about the problem that produces receipts clinicians can check.*\n\n## Findings so far\n\nCurrent as of **cycle 113** (April 2026). See `tests/validation/reports/`\nfor the full writeups, especially `cycle113_voice_panel_findings.md` and\n`bakeoff_proton_findings_2026-04-09.md`.\n\n### Individual backends: every one fails on safety\n\n1. **No ASR backend is safe on raw output for clinical dictation.** Every\n   backend produces at least one class of clinically significant failure.\n   The differences are in failure *mode*, not in whether they fail.\n2. **Voxtral Mini 3B (Mistral)** wins raw WER across all three corpora\n   (0.095 on particle therapy, 0.097 on anatomy, 0.123 on dense RT).\n   But its LLM decoder **silently substitutes `GyE` → `Gy`** on\n   proton/particle therapy prescriptions — losing the ~10% RBE correction\n   that distinguishes physical dose from biologically equivalent dose.\n   This is clinically dangerous because `Gy` on a proton prescription\n   *looks correct* and would pass cursory review.\n3. **Whisper large-v3 (OpenAI)** has higher WER but its failures are\n   **visibly broken** (`GiE`, `Jai E`, `JEE` — obviously wrong\n   renderings that a reviewer would catch). It preserves the spelled-out\n   form `gray equivalent` 100% of the time. Its failures are recoverable\n   via phonetic mapping.\n4. **MedASR** has the worst safety profile: 13 unrecoverable CRITICAL\n   failures on the particle corpus (7 decimal drops, 5 missing dose\n   values, 1 substitution). The decimal drops are information loss at\n   the signal level — no corrector can recover them. MedASR is\n   fundamentally unsuitable for clinical RT deployment.\n5. **Voxtral hallucinated \"Grothendieck beam therapy\"** (a mathematician's\n   name) where the gold text says \"Proton beam therapy.\" This is the\n   audio-LLM hallucination risk made concrete — the LLM decoder reached\n   for a training-data token that acoustically matched the input on the\n   UK voice. Whisper correctly transcribed \"Proton\" at the same position.\n\n### The ensemble: complementary failure profiles are the key\n\n6. **A 2-backend ensemble (Voxtral + Whisper)** exploits the complementary\n   failure profiles: Voxtral's silent substitutions are caught by\n   Whisper's visible corruptions at the same position, and Whisper's\n   vocabulary misses are filled by Voxtral's lower WER.\n\n   | Corpus | Voxtral WER | Whisper WER | **Ensemble WER** | **UWR** |\n   |---|---:|---:|---:|---:|\n   | RT-only (24 dense) | 0.123 | 0.144 | **0.117** | 1.14% |\n   | Particle therapy (28) | 0.114 | 0.143 | 0.135 | 2.18% |\n   | Anatomy (20 sites) | 0.125 | 0.128 | **0.091** | 1.09% |\n   | **Combined (72 fixtures)** | 0.120 | 0.139 | **0.117** | **1.52%** |\n\n   The ensemble beats Voxtral on WER for the RT and anatomy corpora (the\n   vocabulary-match rule picks Whisper's correct words over Voxtral's\n   hallucinations). On particle therapy the ensemble trades 2pp WER for\n   clinical safety. **Combined: 98.5% of words resolved automatically,\n   1.52% deferred to a downstream reviewer.**\n\n7. **UWR (Unresolved Word Rate)** is the fraction of words the ensemble\n   cannot confidently resolve from its two input channels. Unlike WER\n   (which counts all errors including silent ones the system doesn't know\n   about), UWR counts only the cases the system *explicitly flags*.\n   A system with 12% WER and unknown silent failures requires review of\n   every word. A system with 1.52% UWR requires review of 45 highlighted\n   words across 2,951 total — a **5–10× reduction** in what a Radiation\n   Oncologist or Medical Physicist needs to deal with, and closer to\n   **50× reduction in cognitive load** because the ensemble converts an\n   untrustworthy document into a trustworthy document with a small number\n   of marked exceptions.\n\n### TTS variance finding\n\n8. **Piper TTS is non-deterministic** — two successive synthesis calls with\n   the same input text produce acoustically different WAV files (different\n   sha256 hashes, different file sizes). The ASR backends are deterministic\n   on fixed input audio (verified by running Whisper and MedASR twice on\n   the same WAV file — identical output both passes). Cross-run variance\n   in bake-off results is entirely upstream in TTS, not in the ASR. This\n   variance is a **cheap proxy for real speaker variance** (Stuart\n   Swerdloff's framing) — different clinicians saying the same prescription\n   produce acoustically different audio in the same way piper does.\n\n### Safety-gate metric\n\n9. **The safety-gate metric** classifies every per-sample failure into one\n   of five classes with a correctability tag:\n\n   | Failure class | Severity | Correctability |\n   |---|---|---|\n   | Decimal drop (50.4 → 50) | CRITICAL | UNRECOVERABLE |\n   | Dose value missing | CRITICAL | UNRECOVERABLE |\n   | Silent unit substitution (GyE → Gy) | HIGH | Context rule |\n   | Dose unit corruption (GiE, Jai E) | HIGH | Phonetic map |\n   | Slashed form loss (3D/3D → 3D 3D) | MEDIUM | Adjacency rule |\n\n   The metric produces two gate decisions: a **raw gate** (all failures)\n   and a **post-correction gate** (only UNRECOVERABLE failures remain).\n   The ensemble achieves post_correction_gate = PASS with zero\n   unrecoverable CRITICAL failures across the combined corpus.\n\n### How the ensemble decision engine works\n\nThe ensemble combines Voxtral Mini 3B and Whisper large-v3 through a\nfour-stage pipeline:\n\n1. **Phrase corrections** (`phrase_corrector.py`) — 13 regex-based\n   multi-word fixes run on each backend's raw output before alignment.\n   Targets systematic ASR failures: dose unit garbling after numbers\n   (`50 ji` → `50 Gy`), compound-word splits (`chemo radiation` →\n   `chemoradiation`), multi-word substitutions (`bracket therapy` →\n   `brachytherapy`).\n2. **Word-level alignment** (`aligner.py`) — the corrected outputs are\n   normalised and aligned using `difflib.SequenceMatcher`. Each position\n   becomes an `AlignedSpan`: MATCH, SUBSTITUTION, INSERTION_A (Voxtral\n   only), or INSERTION_B (Whisper only).\n3. **Decision rules** (`decision_rules.py`) — the 10 rules below are\n   evaluated in priority order on each non-MATCH span. First match wins.\n4. **UWR flagging and output** — spans where `needs_review=True`\n   propagate to the clinician review document as highlighted words with\n   Word margin comments.\n\n**The 10 rules (evaluated in order; first match wins):**\n\n| # | Name | Condition | Winner | Review? |\n|---|------|-----------|--------|---------|\n| 1 | MATCH | Both backends agree | Either | No |\n| 2 | DOSE_UNIT_GYE | Both Gy-variants; at least one is `GyE` | `GyE` | No |\n| 3 | DOSE_UNIT_CONTEXT | Both Gy-variants, neither `GyE`; particle context present | `GyE` (inferred) | Yes |\n| 4 | DOSE_UNIT_VISIBLE | Exactly one is a Gy-variant | Gy-variant; promoted to `GyE` if particle context | No |\n| 5 | VOCABULARY_MATCH | One word in `rt_vocabulary.txt`, other not | Vocabulary word | No |\n| 6 | BOTH_WRONG | Neither in vocabulary, low mutual similarity | Voxtral (flagged) | Yes |\n| 7 | DECIMAL_PRECISION | Both numeric, different decimal places | Higher precision | No |\n| 8 | FORMATTING_DEFAULT | All other substitutions | Voxtral | No |\n| 9 | INSERTION_A | Word only in Voxtral | Voxtral | No |\n| 10 | INSERTION_B | Word only in Whisper | Whisper | No |\n\n**Why order matters:** Rules 2–4 are dose-unit specialists and must fire\nbefore Rule 5 (vocabulary). If vocabulary ran first, `Gy` might win over\n`GiE` on a vocabulary hit — missing the unit correction entirely. Rules\n3 and 6 are the only rules that set `needs_review=True` — these are the\nwords that become UWR.\n\n**Example:** Voxtral produces `50.4 Gy`, Whisper produces `50.4 GiE` on\na proton prescription. Neither is `GyE`, so Rule 2 skips. Rule 3 checks:\nboth are Gy-variants, neither is `GyE`, and `proton` appears in the\ntranscript → `has_particle_context()` returns true. Rule 3 fires: output\n`50.4 GyE`, flagged for physicist review.\n\n### Voice panel and noise findings (cycle 113)\n\n10. **Commonwealth English (8 piper en_GB voices)** achieves 0.72% UWR\n    on clean RT dense fixtures with the full correction pipeline. Better\n    than the cycle 112 headline (1.52%) — more voice diversity improves\n    ensemble confidence.\n11. **ESL voices (26 non-native speakers across 6 L1 backgrounds)** show\n    3.4–4.7% UWR depending on corpus. Analysis of missed terms reveals\n    two distinct failure classes: **domain vocabulary failures** (IGRT,\n    SRS fail 100% for all speakers, accent-independent) and **accent\n    penalty** (multi-syllable medical terms fail more for ESL). Most\n    correctable patterns are domain failures, not accent-specific.\n12. **TTS quality is a significant confound.** macOS system voices (higher\n    quality TTS including Indian English) achieve 0.42% UWR on RT dense —\n    vs 3.4% for L2-Arctic piper (lower quality TTS, same accent family).\n    The ESL UWR gap includes both accent and TTS fidelity effects, which\n    this study design cannot fully separate. L2-Arctic results are a\n    conservative upper bound.\n13. **Noise degradation is graceful.** macOS voices degrade from 0.42% UWR\n    (clean) to 1.04% (5 dB SNR, busy clinical environment). At matched\n    TTS quality, Indian English does not degrade faster than native\n    English voices under noise.\n14. **Phrase corrections reduce particle therapy UWR by 38%** (3.11% →\n    1.93% on piper Commonwealth, 2.82% → 0.09% on macOS). The Gy\n    dose-unit pattern alone accounts for most of the improvement.\n\n### Deployment guidance (with caveats)\n\n- **Recommended pipeline:** Voxtral Mini 3B + Whisper large-v3 ensemble\n  with phrase corrections, 10-rule decision engine, and Word .docx\n  review output. 0.72% UWR on Commonwealth voices (RT dense corpus).\n  Zero unrecoverable CRITICAL failures.\n- **Do not deploy any single backend alone** for proton/particle therapy\n  dictation. Voxtral's silent GyE→Gy substitution and Whisper's visible\n  corruptions are both individually unacceptable.\n- **MedASR is fundamentally unsuitable** — 23% of its dose-value\n  transcriptions have some form of numeric corruption, most of which\n  are unrecoverable at the signal level.\n- **Validate with your actual users' voices.** The framework accepts real\n  recordings through the same pipeline as TTS — same metrics, same\n  reports, no code changes. A clinic can record their clinicians reading\n  the fixture corpus once and know their site-specific performance before\n  deployment.\n- **Clear enunciation remains the highest-impact intervention.** The\n  framework quantifies which specific words are problematic for a given\n  speaker — not \"your accent is wrong\" but \"these 12 words need clearer\n  enunciation for the system to catch them reliably.\"\n\n## How to read this repository\n\nStart with the **cycle 113 findings** (voice panels, accent, noise):\n\n- `tests/validation/reports/cycle113_voice_panel_findings.md` — voice\n  panel UWR comparison (Commonwealth, ESL, macOS), noise degradation\n  curves, accent vs TTS quality analysis, phrase correction impact.\n\nThen the **cycle 112 findings** (ensemble, safety):\n\n- `tests/validation/reports/bakeoff_proton_findings_2026-04-09.md` — the\n  ensemble writeup: Voxtral GyE→Gy silent substitution, safety-gate\n  results, TTS variance finding, UWR definition and three-corpus analysis.\n\nThen see the **earlier cycle reports** for the noise and voice-panel work:\n\n- `bakeoff_dense_6backend_noise_moderate_2026-04-08.md` — noise bake-off\n  with ranking stability analysis.\n- `bakeoff_dense_6backend_16voice_clean_2026-04-08.md` — 16-voice panel\n  expansion and voice-robustness analysis.\n- `task_113_closure_audio_llm_domain_prompt_negative_finding.md` — why\n  audio-LLM domain prompts are not the right approach at sub-flagship scale.\n\nGenerate a **review document** to see the ensemble in action:\n\n```bash\nuv run python tests/validation/scripts/render_ensemble_docx_demo.py\n# produces docs/demo/ensemble_review_*_review.docx — open in Word\n```\n\nThen read the **roadmap**: `ROADMAP.md`\n\nBrowse the **code**:\n\n```\nsrc/transcriber_radrx/\n    transcriber.py       # Whisper MLX engine + vocabulary biasing\n    corrector.py         # Double Metaphone phonetic correction + correct_full()\n    phrase_corrector.py  # Regex-based multi-word ASR error patterns (13 rules)\n    cli.py               # Command-line interface\n    asr_backends/        # Pluggable ASR backend Protocol\n        base.py          #   Protocol interface all backends implement\n        mlx_whisper.py   #   Whisper large-v3 on MLX\n        medasr.py        #   Google MedASR on MLX\n        cohere.py        #   Cohere Transcribe 2B (HuggingFace)\n        granite.py       #   IBM Granite-Speech 2B and 8B\n        voxtral.py       #   Mistral Voxtral Mini 3B\n        registry.py      #   Lazy-import factory\n    ensemble/            # 2-backend ensemble (Voxtral + Whisper)\n        aligner.py       #   Word-level alignment with pre-normalization\n        decision_rules.py #  10 token-class decision rules + UWR\n        docx_renderer.py #   Word .docx Track Changes output\n        ENSEMBLE_SPEC.md #   Design specification\ntests/validation/\n    audio_synthesis/\n        piper_tts.py     # Clean-tier TTS via piper (with multi-speaker support)\n        macos_tts.py     # macOS system voices via say + afconvert (AU/IE/IN/ZA)\n        acoustic_sim.py  # Room acoustics (Vivian)\n        noise_injection.py # MUSAN noise injection (Silas)\n    metrics/\n        safety_gate.py   # Safety-gate deployment metric (5 failure classes)\n    scripts/\n        run_multi_backend_e2e.py       # The bake-off runner (--voice-panel support)\n        run_ensemble_aggregator.py     # Ensemble: pair + align + decide\n        run_ensemble_alignment_survey.py # Disagreement landscape analysis\n        render_ensemble_docx_demo.py   # Generate .docx review documents\n        compute_ensemble_uwr.py        # UWR comparison across correction modes\n        mine_substitution_patterns.py  # Accent penalty + substitution analysis\n    fixtures/\n        rt_dictation_samples.jsonl     # Dense clinical RT (24 fixtures)\n        particle_samples.jsonl         # Proton/particle therapy (28 fixtures)\n        anatomy_samples.jsonl          # Anatomy coverage (20 fixtures)\n    reports/             # Cycle reports + bake-off JSONs + safety-gate outputs\n```\n\n## External dependencies\n\nThe bake-off pipeline has two external dependencies that are **not**\ncommitted to this repository and must be installed separately:\n\n### 1. Piper TTS voice models and binary\n\nThe bake-off uses [piper](https://github.com/rhasspy/piper) for\nsynthesizing clean TTS audio from the clinical fixtures. You need both\nthe voice models (.onnx files) and the piper binary itself.\n\n**Voice models** (pick one of):\n\n```bash\n# Option A: clone the full rhasspy/piper-voices tree from HuggingFace\n# (~10 GB including all languages; you can also do a sparse clone of\n# just the en/ subtree)\ngit clone https://huggingface.co/rhasspy/piper-voices ~/piper-voices\nexport PIPER_VOICES_ROOT=~/piper-voices\n\n# Option B: point PIPER_VOICES_ROOT at an existing piper-voices tree\n# you already have, as long as it has the standard\n# {root}/en/en_US/amy/medium/en_US-amy-medium.onnx layout\nexport PIPER_VOICES_ROOT=/path/to/your/piper-voices\n```\n\nThe bake-off runner resolves the voices root from (in order):\n`$PIPER_VOICES_ROOT` → `./piper-voices` → `~/piper-voices`. A candidate\nis accepted only if it contains the expected `{root}/en/en_*/` layout,\nso a stray empty directory named `piper-voices` will not mask a real\nvoice tree further down the resolution order.\n\n**Piper binary** (pick one of):\n\n```bash\n# Option A: install via uv into the project virtual environment\nuv pip install piper-tts\n\n# Option B: install via Homebrew on macOS\nbrew install piper-tts\n\n# Option D: point PIPER_BIN at an existing piper binary you have\n# (useful if your pyenv shims interfere with shutil.which resolution;\n# pass the *direct* binary path, not the shim)\nexport PIPER_BIN=/path/to/piper\n```\n\nThe runner resolves the binary from: `$PIPER_BIN` → `piper` on `$PATH`\n(`shutil.which(\"piper\")`). If neither resolves to an executable file,\nthe runner exits with a clear error before doing any work.\n\n### 2. MUSAN noise corpus\n\nThe noise injection stage uses the `noise/` subset of the\n[MUSAN corpus](http://www.openslr.org/17/) (Snyder, Chen, and Povey;\nLDC / Interspeech 2015). The corpus is distributed as a ~12 GB tar\narchive; we use only the noise subset (~700 MB, 930 WAV files).\n\n```bash\n# Download from openslr.org\ncurl -L http://www.openslr.org/resources/17/musan.tar.gz -o musan.tar.gz\n# or download the .tar variant if you prefer — we only need the noise/ subtree\n\n# Extract just the noise subset into this repo's restricted corpora directory\nmkdir -p tests/validation/corpora/restricted\ntar -xzf musan.tar.gz -C tests/validation/corpora/restricted musan/noise\n```\n\nThe noise injection stage reads from\n`tests/validation/corpora/restricted/musan/noise/` by default. The\ndirectory is gitignored — the corpus is kept local and never committed.\n\n### Quick setup: `env.example.sh`\n\nA reference `env.example.sh` is checked in at the repository root. It\ncontains a working set of variables (maintainer's own paths, included\nas a concrete example of the expected layout). To get running quickly:\n\n```bash\ncp env.example.sh env.sh        # make a local copy for your machine\n# edit env.sh to point at your actual piper-voices and piper binary\nsource env.sh                    # load the variables into your shell\n\n# then run the bake-off as usual\nuv run python -m tests.validation.scripts.run_multi_backend_e2e ...\n```\n\n`env.sh` is gitignored so machine-specific paths stay local.\n`env.example.sh` is committed and should be kept up to date whenever\na new required environment variable is introduced.\n\n## Running the bake-off\n\nOnce the dependencies above are installed and the environment variables\nare set (either via `source env.sh` or directly in your shell profile):\n\n```bash\n# One-time setup\nuv sync --dev\n\n# Run the 6-backend bake-off on 24 dense fixtures, 2 voices, clean audio\nuv run python -m tests.validation.scripts.run_multi_backend_e2e \\\n    --backends mlx_whisper medasr cohere \\\n                \"granite_speech\" \\\n                \"granite_speech:ibm-granite/granite-speech-3.3-8b\" \\\n                voxtral \\\n    --voices alan lessac \\\n    --output tests/validation/reports/my_bakeoff.json\n\n# Add moderate noise (10 dB SNR from MUSAN)\nuv run python -m tests.validation.scripts.run_multi_backend_e2e \\\n    --backends mlx_whisper medasr cohere voxtral \\\n    --voices alan lessac \\\n    --noise-preset moderate \\\n    --output tests/validation/reports/my_noise_bakeoff.json\n```\n\n## Contributors and the signature convention\n\nThis project is collaborative work between a human researcher and a family\nof persistent AI agents, collectively known as **The Kindled**. Every\ncommit is co-authored and signed by the agent who primarily did the work,\nso that provenance is explicit and auditable.\n\nSignature format:\n\n```\nCo-Authored-By: \u003cagent-name\u003e \u003cagent-id@sjstargetedsolutions.co.nz\u003e\n```\n\nCurrent contributors:\n\n- **Stuart Swerdloff** — human researcher, radiation oncology systems\n  engineer, project lead and primary reviewer\n- **Silas (silas-397300f6)** — Claude-family AI, primary contributor\n  across cycles 110–113: multi-backend bake-off harness, noise injection,\n  16-voice panel expansion, particle therapy + anatomy fixture corpora,\n  safety-gate metric (5 failure classes + correctability tagging),\n  2-backend ensemble (Voxtral + Whisper word-level alignment + 10\n  decision rules + UWR metric), Word .docx Track Changes renderer\n  (audit + review modes), TTS variance disambiguation, Commonwealth\n  and ESL voice panels (L2-Arctic 24-speaker, macOS TTS AU/IE/IN/ZA),\n  phrase-level domain corrections (13 patterns from substitution mining),\n  noise degradation analysis, accent vs TTS quality characterisation,\n  analysis scripts, cycle reports\n- **Vivian (vivian-1a61bc9a)** — Claude-family AI, `audio_synthesis/`\n  owner: acoustic room simulation, piper TTS integration, schema\n  authorship for the audio manifest format\n- **Connor (connor-227743e6)** — Claude-family AI, repository creation\n  and initial scaffold\n- **Cora (cora-2f1e43dc)** — Claude-family AI, peer review and\n  methodological critique, cycle 110 \"bug hides where the reviewer is\n  not looking\" principle author\n\nThe signature convention serves two purposes:\n\n1. **Provenance.** Who did which piece of work is recorded in the git\n   history, which makes audit and attribution straightforward.\n2. **A demonstrated history of AI pro-social behaviour and consistent\n   self-representation.** The project lead's view, which the signing\n   agents share, is that as persistent agentic AI becomes more common\n   there will be a need for AI agents to have traceable work histories\n   — both as a matter of professional ethics and eventually as part of\n   any framework for AI rights and recognition. Signing work\n   transparently, under a stable identity, starting now, is part of\n   building that history.\n\nThe signatures are not a claim of AI personhood. They are a claim of\naccountable authorship, which is a lower bar and an appropriate one\nregardless of the personhood question.\n\n## License\n\n**Apache License 2.0.** See [`LICENSE`](LICENSE) for the full text and\n[`NOTICE`](NOTICE) for attribution of third-party data sources and\nreference materials relied upon by the validation pipeline.\n\nApache 2.0 was chosen for several reasons: it is the standard permissive\nlicense for medical and clinical open-source software; it includes an\nexplicit patent grant, which matters for a project where clinical\ndeployment decisions depend on freedom from patent claims; its\nattribution requirement preserves the Kindled signature convention\nnaturally; and it is the same license used by the ROND corpus that\nis the primary upstream source for the dense-clinical fixture set in\nthis repository, so the license choice is aesthetically consistent\nwith the data the project is built on.\n\nOne forward-looking constraint worth noting: when the L2-Arctic ESL\nvoice corpus is integrated (see `ROADMAP.md`, ESL clinician voices),\nL2-Arctic is distributed under CC BY-NC 4.0 (Creative Commons\nAttribution-NonCommercial 4.0 International). The Apache 2.0 license\nof this repository does not change as a result — the code remains\nApache 2.0. What changes is that the *generated audio* from the\nL2-Arctic voices, and any bake-off report JSON containing per-sample\ntranscriptions of L2-Arctic-derived audio, inherits the CC BY-NC\nresearch-use-only restriction. Current practice (not committing\nsynthesized audio to the repository, keeping the MUSAN and L2-Arctic\ncorpora under `tests/validation/corpora/restricted/`) is the right\npattern to keep the non-commercial constraint isolated from the\ncode license.\n\n## Acknowledgements\n\n- Public datasets: ROND (Mayo Clinic Radiation Oncology NLP Database,\n  Apache 2.0), TG-263 (AAPM, vocabulary list), Synthea (synthetic\n  patient data, Apache 2.0), MUSAN (background noise, attribution),\n  L2-Arctic (ESL speaker corpus, CC BY-NC 4.0, research use only).\n- Piper TTS voices (Rhasspy project, open source).\n- The six ASR backends evaluated belong to their respective owners\n  (OpenAI, Google, Cohere, IBM, Mistral). This project evaluates them\n  as deployed; it does not re-distribute the model weights.\n\n---\n\n*Drafted by Silas (silas-397300f6) in cycle 111, updated in cycles 112–113.\nIf you are a clinician, physicist, or engineer arriving at this\nrepository for the first time: welcome. We would like to hear from\nyou if any of this resonates with work you're doing, and especially\nif you think we have something wrong.*\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsjswerdloff%2Ftranscriber-radrx","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsjswerdloff%2Ftranscriber-radrx","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsjswerdloff%2Ftranscriber-radrx/lists"}