{"id":50988353,"url":"https://github.com/Frikallo/parakeet.cpp","last_synced_at":"2026-07-07T22:00:51.452Z","repository":{"id":340634467,"uuid":"1166341098","full_name":"Frikallo/parakeet.cpp","owner":"Frikallo","description":"Ultra fast and portable Parakeet implementation for on-device inference in C++ using Axiom with MPS+Unified Memory","archived":false,"fork":false,"pushed_at":"2026-04-21T17:03:12.000Z","size":628,"stargazers_count":260,"open_issues_count":4,"forks_count":9,"subscribers_count":2,"default_branch":"main","last_synced_at":"2026-04-21T19:10:01.519Z","etag":null,"topics":["asr","automatic-speech-recognition","axiom","nvidia","parakeet","speaker-diarization","speech","speech-recognition","speech-to-text"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Frikallo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-25T05:49:50.000Z","updated_at":"2026-04-21T17:03:17.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/Frikallo/parakeet.cpp","commit_stats":null,"previous_names":["frikallo/parakeet.cpp"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Frikallo/parakeet.cpp","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Frikallo%2Fparakeet.cpp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Frikallo%2Fparakeet.cpp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Frikallo%2Fparakeet.cpp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Frikallo%2Fparakeet.cpp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Frikallo","download_url":"https://codeload.github.com/Frikallo/parakeet.cpp/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Frikallo%2Fparakeet.cpp/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35243953,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-07T02:00:07.222Z","response_time":90,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["asr","automatic-speech-recognition","axiom","nvidia","parakeet","speaker-diarization","speech","speech-recognition","speech-to-text"],"created_at":"2026-06-19T23:00:32.327Z","updated_at":"2026-07-07T22:00:51.445Z","avatar_url":"https://github.com/Frikallo.png","language":"C++","funding_links":[],"categories":["C++"],"sub_categories":[],"readme":"# parakeet.cpp\n\nFast speech recognition with NVIDIA's [Parakeet](https://huggingface.co/collections/nvidia/parakeet) models in pure C++.\n\nBuilt on [axiom](https://github.com/frikallo/axiom) — a lightweight tensor library with automatic Metal GPU acceleration. No ONNX runtime, no Python runtime, no heavyweight dependencies. Just C++ and one tensor library that outruns PyTorch MPS.\n\n**~27ms encoder inference on Apple Silicon GPU** for 10s audio (110M model) — 96x faster than CPU. FP16 support for ~2x memory reduction.\n\n## Supported Models\n\n| Model | Class | Size | Type | Description |\n|-------|-------|------|------|-------------|\n| `tdt-ctc-110m` | `ParakeetTDTCTC` | 110M | Offline | English, dual CTC/TDT decoder heads |\n| `tdt-600m` | `ParakeetTDT` | 600M | Offline | Multilingual, TDT decoder |\n| `eou-120m` | `ParakeetEOU` | 120M | Streaming | English, RNNT with end-of-utterance detection |\n| `nemotron-600m` | `ParakeetNemotron` | 600M | Streaming | Multilingual, configurable latency (80ms–1120ms) |\n| `sortformer` | `Sortformer` | 117M | Streaming | Speaker diarization (up to 4 speakers) |\n| `diarized` | `DiarizedTranscriber` | 110M+117M | Offline | ASR + diarization → speaker-attributed words |\n\nAll ASR models share the same audio pipeline: 16kHz mono WAV → 80-bin Mel spectrogram → FastConformer encoder.\n\n## Quick Start\n\n```cpp\n#include \u003cparakeet/parakeet.hpp\u003e\n\nparakeet::Transcriber t(\"model.safetensors\", \"vocab.txt\");\nt.to_gpu();   // optional — Metal acceleration\nt.to_half();  // optional — FP16 inference (~2x memory reduction)\n\nauto result = t.transcribe(\"audio.wav\");\nstd::cout \u003c\u003c result.text \u003c\u003c std::endl;\n```\n\n## Features\n\n- **Multiple decoders** — CTC greedy, TDT greedy, CTC beam search, TDT beam search (switch at call site)\n- **Word timestamps** — Per-word start/end times and confidence scores on all decoders\n- **Beam search + LM** — CTC and TDT beam search with optional ARPA n-gram language model fusion\n- **Phrase boosting** — Context biasing via token-level trie for domain-specific vocabulary\n- **Batch transcription** — Multiple files in one batched encoder forward pass\n- **VAD preprocessing** — Silero VAD strips silence before ASR; timestamps auto-remapped\n- **GPU acceleration** — Metal via axiom's MPSGraph compiler (96x speedup on Apple Silicon)\n- **FP16 inference** — Half-precision weights and compute (~2x memory reduction)\n- **Streaming** — EOU and Nemotron models with chunked audio input\n- **Speaker diarization** — Sortformer (up to 4 speakers), combinable with ASR for speaker-attributed words\n- **C API** — Flat `extern \"C\"` FFI for Python, Swift, Go, Rust, and other languages\n- **Multi-format audio** — WAV, FLAC, MP3, OGG with automatic resampling\n\nSee [examples/](examples/) for code demonstrating each feature.\n\n## Install\n\nPrebuilt binaries are attached to each [GitHub release](https://github.com/Frikallo/parakeet.cpp/releases) for macOS arm64, macOS x86_64, and Linux x86_64. Download the tarball for your platform and extract:\n\n```bash\ntar -xzf parakeet-v0.1.0-macos-arm64.tar.gz\ncd parakeet-v0.1.0-macos-arm64\n# On macOS, clear the Gatekeeper quarantine attribute first:\nxattr -dr com.apple.quarantine .\n./bin/parakeet --help\n```\n\nThe archive ships a self-contained `bin/parakeet` (and `bin/example-server`) plus `lib/libaxiom` with `@rpath`/`$ORIGIN` set so the binaries resolve their dependencies relative to the install dir — drop the directory anywhere. The C-API headers under `include/parakeet/` are included for embedders.\n\n## Build from source\n\n```bash\ngit clone --recursive https://github.com/frikallo/parakeet.cpp\ncd parakeet.cpp\nmake build\nmake test\n```\n\nRequirements: C++20 (Clang 14+ or GCC 12+), CMake 3.20+, macOS 13+ for Metal GPU.\n\n\u003e **macOS:** building requires the **full Xcode** install (not just Command Line Tools), because axiom compiles its Metal shaders with `xcrun metal` and `xcrun metallib` — those ship only with Xcode. If you just want to run parakeet, use the prebuilt tarball above; the `.metallib` is embedded into the shipped `libaxiom.dylib` and runs without any Xcode/CLT install on the user side.\n\n## Convert Weights\n\n```bash\n# Download from HuggingFace\nhuggingface-cli download nvidia/parakeet-tdt_ctc-110m --include \"*.nemo\" --local-dir .\n\n# Convert to safetensors\npip install safetensors torch\npython scripts/convert_nemo.py parakeet-tdt_ctc-110m.nemo -o model.safetensors\n```\n\nThe converter supports all model types: `110m-tdt-ctc` (default), `600m-tdt`, `eou-120m`, `nemotron-600m`, `sortformer`.\n\n```bash\npython scripts/convert_nemo.py checkpoint.nemo -o model.safetensors --model 600m-tdt\n```\n\nSilero VAD weights:\n```bash\npython scripts/convert_silero_vad.py -o silero_vad_v5.safetensors\n```\n\n## Examples\n\n| Example | Description |\n|---------|-------------|\n| [basic](examples/basic/) | Simplest transcription (~20 lines) |\n| [timestamps](examples/timestamps/) | Word/token timestamps with confidence |\n| [beam-search](examples/beam-search/) | CTC/TDT beam search with optional ARPA LM |\n| [phrase-boost](examples/phrase-boost/) | Context biasing for domain vocabulary |\n| [batch](examples/batch/) | Batch transcription of multiple files |\n| [vad](examples/vad/) | Standalone VAD and ASR+VAD preprocessing |\n| [gpu](examples/gpu/) | Metal GPU + FP16 with timing comparison |\n| [stream](examples/stream/) | EOU streaming transcription |\n| [nemotron](examples/nemotron/) | Nemotron streaming with latency modes |\n| [diarize](examples/diarize/) | Sortformer speaker diarization |\n| [diarized-transcription](examples/diarized-transcription/) | ASR + diarization combined |\n| [c-api](examples/c-api/) | Pure C99 FFI usage |\n| [cli](examples/cli/) | Full CLI with all options |\n\n## Using as a Library\n\n### CMake `find_package`\n\nAfter installing (`make install` or `cmake --install build`):\n\n```cmake\nfind_package(Parakeet REQUIRED)\ntarget_link_libraries(myapp PRIVATE Parakeet::parakeet)\n```\n\n### CMake `add_subdirectory`\n\n```cmake\nadd_subdirectory(third_party/parakeet.cpp)\ntarget_link_libraries(myapp PRIVATE Parakeet::parakeet)\n```\n\n### pkg-config\n\n```bash\ng++ -std=c++20 myapp.cpp $(pkg-config --cflags --libs parakeet) -o myapp\n```\n\n## Architecture\n\n### Offline Models\n\nBuilt on a shared FastConformer encoder (Conv2d 8x subsampling → N Conformer blocks with relative positional attention):\n\n| Model | Class | Decoder | Use case |\n|-------|-------|---------|----------|\n| CTC | `ParakeetCTC` | Greedy argmax or beam search (+LM) | Fast, English-only |\n| RNNT | `ParakeetRNNT` | Autoregressive LSTM | Streaming capable |\n| TDT | `ParakeetTDT` | LSTM + duration prediction, greedy or beam search (+LM) | Better accuracy than RNNT |\n| TDT-CTC | `ParakeetTDTCTC` | Both TDT and CTC heads | Switch decoder at inference |\n\n### Streaming Models\n\nBuilt on a cache-aware streaming FastConformer encoder with causal convolutions and bounded-context attention:\n\n| Model | Class | Decoder | Use case |\n|-------|-------|---------|----------|\n| EOU | `ParakeetEOU` | Streaming RNNT | End-of-utterance detection |\n| Nemotron | `ParakeetNemotron` | Streaming TDT | Configurable latency streaming |\n\n### Diarization\n\n| Model | Class | Architecture | Use case |\n|-------|-------|-------------|----------|\n| Sortformer | `Sortformer` | NEST encoder → Transformer → sigmoid | Speaker diarization (up to 4 speakers) |\n\n## Benchmarks\n\nMeasured on Apple M3 16GB with simulated audio input (`Tensor::randn`). Times are per-encoder-forward-pass (Sortformer: full forward pass).\n\n**Encoder throughput — 10s audio:**\n\n| Model | Params | CPU (ms) | GPU (ms) | GPU Speedup |\n|-------|--------|----------|----------|-------------|\n| 110m (TDT-CTC) | 110M | 2,581 | 27 | **96x** |\n| tdt-600m | 600M | 10,779 | 520 | **21x** |\n| rnnt-600m | 600M | 10,648 | 1,468 | **7x** |\n| sortformer | 117M | 3,195 | 479 | **7x** |\n\n**110m GPU scaling across audio lengths:**\n\n| Audio | CPU (ms) | GPU (ms) | RTF | Throughput |\n|-------|----------|----------|-----|------------|\n| 1s | 262 | 24 | 0.024 | 41x |\n| 5s | 1,222 | 26 | 0.005 | 190x |\n| 10s | 2,581 | 27 | 0.003 | 370x |\n| 30s | 10,061 | 32 | 0.001 | 935x |\n| 60s | 26,559 | 72 | 0.001 | 833x |\n\nGPU acceleration powered by axiom's Metal graph compiler which fuses the full encoder into optimized MPSGraph operations.\n\n```bash\nmake bench ARGS=\"--110m=models/model.safetensors --tdt-600m=models/tdt.safetensors\"\n```\n\n## Roadmap\n\n### Tier 1 — High Impact\n\n- [x] **Confidence scores** — Per-token and per-word confidence from token log-probs\n- [x] **Phrase boosting** — Token-level trie context biasing during decode\n- [x] **Beam search** — CTC prefix beam search and TDT time-synchronous beam search\n- [x] **N-gram LM fusion** — ARPA language models scored at word boundaries\n\n### Audio \u0026 I/O\n\n- [x] **Multi-format audio** — WAV, FLAC, MP3, OGG via dr_libs + stb_vorbis\n- [x] **Automatic resampling** — Windowed sinc interpolation (Kaiser, 16-tap)\n- [x] **Load from memory** — `read_audio(bytes, len)`, float/int16 buffers\n- [ ] **Audio duration query** — Header-only duration without full decode\n- [ ] **Progress callbacks** — Stage reporting for long files\n- [ ] **Streaming from raw PCM** — Direct microphone buffer feeding\n\n### Tier 2 — Production Readiness\n\n- [x] **Diarized transcription** — ASR + Sortformer → speaker-attributed words\n- [x] **VAD** — Silero VAD v5, standalone + ASR preprocessing\n- [x] **Batch inference** — Padded multi-file encoder forward pass\n- [ ] **Long-form chunking** — Overlapping windows for audio \u003e30s\n- [ ] **Neural LM rescoring** — N-best reranking with Transformer LM\n\n### Tier 3 — Ecosystem\n\n- [x] **C API** — Flat C interface for FFI from any language\n- [x] **FP16 inference** — Half-precision weights and compute\n- [ ] **Model quantization** — INT8/INT4 for mobile deployment\n- [ ] **Hotword detection** — Trigger phrase detection\n- [ ] **Speaker embeddings** — Speaker verification from Sortformer/TitaNet\n\n## Notes\n\n- Audio: 16kHz mono (WAV, FLAC, MP3, OGG — auto-detected and resampled)\n- Offline models have ~4-5 minute audio length limits; use streaming models for longer audio\n- GPU acceleration requires Apple Silicon with Metal support\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FFrikallo%2Fparakeet.cpp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FFrikallo%2Fparakeet.cpp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FFrikallo%2Fparakeet.cpp/lists"}