{"id":16612991,"url":"https://github.com/ttys3/cld3-rs","last_synced_at":"2026-04-16T14:43:06.273Z","repository":{"id":173971340,"uuid":"651383925","full_name":"ttys3/cld3-rs","owner":"ttys3","description":"Compact Language Detector v3 (CLD3) Rust binding","archived":false,"fork":false,"pushed_at":"2023-06-09T15:50:45.000Z","size":715,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-25T23:41:28.033Z","etag":null,"topics":["language","language-detection","language-detector","whatlang"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ttys3.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-09T05:58:33.000Z","updated_at":"2025-05-20T10:58:08.000Z","dependencies_parsed_at":null,"dependency_job_id":"4a947f17-802e-46a2-b3ec-9b1a04b994ce","html_url":"https://github.com/ttys3/cld3-rs","commit_stats":null,"previous_names":["ttys3/cld3-rs"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/ttys3/cld3-rs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ttys3%2Fcld3-rs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ttys3%2Fcld3-rs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ttys3%2Fcld3-rs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ttys3%2Fcld3-rs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ttys3","download_url":"https://codeload.github.com/ttys3/cld3-rs/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ttys3%2Fcld3-rs/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31891038,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-16T11:36:10.202Z","status":"ssl_error","status_checked_at":"2026-04-16T11:36:09.652Z","response_time":69,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["language","language-detection","language-detector","whatlang"],"created_at":"2024-10-12T01:45:40.623Z","updated_at":"2026-04-16T14:43:06.236Z","avatar_url":"https://github.com/ttys3.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"## cld3-rs\n\n[![crates.io](https://badgen.net/crates/v/cld3)](https://crates.io/crates/cld3)\n[![docs.rs](https://docs.rs/cld3/badge.svg)](https://docs.rs/cld3)\n![downloads](https://badgen.net/crates/d/cld3)\n![latest version downloads](https://badgen.net/crates/dl/cld3)\n\nRust binding for Compact Language Detector v3 (CLD3), a neural network model for language identification.\n\nthe C++ code is from https://github.com/google/cld3\n\n## Usage\n\n`libprotobuf` is required to build this crate.\n\n```rust\nlet mut cld = cld3::NNetLanguageIdentifier::new(0, 500).unwrap();\nlet rs = cld.find_language(\"こんにちは\");\nprintln!(\"{:?}\", rs.language);\n// ja\n```\n### Compact Language Detector v3 (CLD3)\n\n* [Model](#model)\n* [Supported Languages](#supported-languages)\n\n### Model\n\nCLD3 is a neural network model for language identification. This package\ncontains the inference code and a trained model. The inference code\nextracts character ngrams from the input text and computes the fraction\nof times each of them appears. For example, as shown in the figure below,\nif the input text is \"banana\", then one of the extracted trigrams is \"ana\"\nand the corresponding fraction is 2/4. The ngrams are hashed down to an id\nwithin a small range, and each id is represented by a dense embedding vector\nestimated during training.\n\nThe model averages the embeddings corresponding to each ngram type according\nto the fractions, and the averaged embeddings are concatenated to produce\nthe embedding layer. The remaining components of the network are a hidden\n(Rectified linear) layer and a softmax layer.\n\nTo get a language prediction for the input text, we simply perform a forward\npass through the network.\n\n![Figure](https://github.com/google/cld3/raw/master/model.png \"CLD3\")\n\n### Supported Languages\n\nThe model outputs BCP-47-style language codes, shown in the table below. For\nsome languages, output is differentiated by script. Language and script names\nfrom\n[Unicode CLDR](https://github.com/unicode-cldr/cldr-localenames-modern/blob/master/main/en).\n\nOutput Code | Language Name   | Script Name\n----------- | --------------- | ------------------------------------------\naf          | Afrikaans       | Latin\nam          | Amharic         | Ethiopic\nar          | Arabic          | Arabic\nbg          | Bulgarian       | Cyrillic\nbg-Latn     | Bulgarian       | Latin\nbn          | Bangla          | Bangla\nbs          | Bosnian         | Latin\nca          | Catalan         | Latin\nceb         | Cebuano         | Latin\nco          | Corsican        | Latin\ncs          | Czech           | Latin\ncy          | Welsh           | Latin\nda          | Danish          | Latin\nde          | German          | Latin\nel          | Greek           | Greek\nel-Latn     | Greek           | Latin\nen          | English         | Latin\neo          | Esperanto       | Latin\nes          | Spanish         | Latin\net          | Estonian        | Latin\neu          | Basque          | Latin\nfa          | Persian         | Arabic\nfi          | Finnish         | Latin\nfil         | Filipino        | Latin\nfr          | French          | Latin\nfy          | Western Frisian | Latin\nga          | Irish           | Latin\ngd          | Scottish Gaelic | Latin\ngl          | Galician        | Latin\ngu          | Gujarati        | Gujarati\nha          | Hausa           | Latin\nhaw         | Hawaiian        | Latin\nhi          | Hindi           | Devanagari\nhi-Latn     | Hindi           | Latin\nhmn         | Hmong           | Latin\nhr          | Croatian        | Latin\nht          | Haitian Creole  | Latin\nhu          | Hungarian       | Latin\nhy          | Armenian        | Armenian\nid          | Indonesian      | Latin\nig          | Igbo            | Latin\nis          | Icelandic       | Latin\nit          | Italian         | Latin\niw          | Hebrew          | Hebrew\nja          | Japanese        | Japanese\nja-Latn     | Japanese        | Latin\njv          | Javanese        | Latin\nka          | Georgian        | Georgian\nkk          | Kazakh          | Cyrillic\nkm          | Khmer           | Khmer\nkn          | Kannada         | Kannada\nko          | Korean          | Korean\nku          | Kurdish         | Latin\nky          | Kyrgyz          | Cyrillic\nla          | Latin           | Latin\nlb          | Luxembourgish   | Latin\nlo          | Lao             | Lao\nlt          | Lithuanian      | Latin\nlv          | Latvian         | Latin\nmg          | Malagasy        | Latin\nmi          | Maori           | Latin\nmk          | Macedonian      | Cyrillic\nml          | Malayalam       | Malayalam\nmn          | Mongolian       | Cyrillic\nmr          | Marathi         | Devanagari\nms          | Malay           | Latin\nmt          | Maltese         | Latin\nmy          | Burmese         | Myanmar\nne          | Nepali          | Devanagari\nnl          | Dutch           | Latin\nno          | Norwegian       | Latin\nny          | Nyanja          | Latin\npa          | Punjabi         | Gurmukhi\npl          | Polish          | Latin\nps          | Pashto          | Arabic\npt          | Portuguese      | Latin\nro          | Romanian        | Latin\nru          | Russian         | Cyrillic\nru-Latn     | Russian         | English\nsd          | Sindhi          | Arabic\nsi          | Sinhala         | Sinhala\nsk          | Slovak          | Latin\nsl          | Slovenian       | Latin\nsm          | Samoan          | Latin\nsn          | Shona           | Latin\nso          | Somali          | Latin\nsq          | Albanian        | Latin\nsr          | Serbian         | Cyrillic\nst          | Southern Sotho  | Latin\nsu          | Sundanese       | Latin\nsv          | Swedish         | Latin\nsw          | Swahili         | Latin\nta          | Tamil           | Tamil\nte          | Telugu          | Telugu\ntg          | Tajik           | Cyrillic\nth          | Thai            | Thai\ntr          | Turkish         | Latin\nuk          | Ukrainian       | Cyrillic\nur          | Urdu            | Arabic\nuz          | Uzbek           | Latin\nvi          | Vietnamese      | Latin\nxh          | Xhosa           | Latin\nyi          | Yiddish         | Hebrew\nyo          | Yoruba          | Latin\nzh          | Chinese         | Han (including Simplified and Traditional)\nzh-Latn     | Chinese         | Latin\nzu          | Zulu            | Latin\n\n\n## similar works\n\nNatural language detection library for Rust.  https://github.com/greyblake/whatlang-rs\n\nTranslation of Google's cld3 language detector into Rust. Just for fun. https://github.com/edvakf/langdetect","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fttys3%2Fcld3-rs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fttys3%2Fcld3-rs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fttys3%2Fcld3-rs/lists"}