{"id":21232482,"url":"https://github.com/tugascript/keyword-extraction-rs","last_synced_at":"2025-07-10T17:31:11.997Z","repository":{"id":158321150,"uuid":"622318083","full_name":"tugascript/keyword-extraction-rs","owner":"tugascript","description":"Keyword extraction algorithms in Rust","archived":false,"fork":false,"pushed_at":"2024-10-12T02:55:05.000Z","size":272,"stargazers_count":19,"open_issues_count":0,"forks_count":3,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-06-20T09:06:57.999Z","etag":null,"topics":["keyword-extraction","nlp"],"latest_commit_sha":null,"homepage":"https://crates.io/crates/keyword_extraction","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"lgpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tugascript.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-01T18:50:17.000Z","updated_at":"2025-05-21T06:37:09.000Z","dependencies_parsed_at":"2023-12-12T09:38:49.387Z","dependency_job_id":"b36a9104-9797-46f7-8f92-a6a87090e74c","html_url":"https://github.com/tugascript/keyword-extraction-rs","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/tugascript/keyword-extraction-rs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tugascript%2Fkeyword-extraction-rs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tugascript%2Fkeyword-extraction-rs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tugascript%2Fkeyword-extraction-rs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tugascript%2Fkeyword-extraction-rs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tugascript","download_url":"https://codeload.github.com/tugascript/keyword-extraction-rs/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tugascript%2Fkeyword-extraction-rs/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262119395,"owners_count":23261898,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["keyword-extraction","nlp"],"created_at":"2024-11-20T23:52:16.171Z","updated_at":"2025-07-10T17:31:11.568Z","avatar_url":"https://github.com/tugascript.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Rust Keyword Extraction\n\n## Introduction\n\nThis is a simple NLP library with a list of unsupervised keyword extraction algorithms:\n\n- Tokenizer for tokenizing text;\n- TF-IDF for calculating the importance of a word in one or more documents;\n- Co-occurrence for calculating relationships between words within a specific window size;\n- RAKE for extracting key phrases from a document;\n- TextRank for extracting keywords and key phrases from a document;\n- YAKE for extracting keywords with a n-gram size (defaults to 3) from a document.\n\n## Algorithms\n\nThe full list of the algorithms in this library:\n\n- Helper algorithms:\n  - [x] Tokenizer\n  - [x] Co-occurrence\n- Keyword extraction algorithms:\n  - [x] TF-IDF\n  - [x] RAKE\n  - [x] TextRank\n  - [x] YAKE\n\n## Usage\n\nAdd the library to your `Cargo.toml`:\n\n```toml\n[dependencies]\nkeyword_extraction = \"1.5.0\"\n```\n\nOr use cargo add:\n\n```bash\ncargo add keyword_extraction\n```\n\n### Features\n\nIt is possible to enable or disable features:\n\n- `\"tf_idf\"`: TF-IDF algorithm;\n- `\"rake\"`: RAKE algorithm;\n- `\"text_rank\"`: TextRank algorithm;\n- `\"yake\"`: YAKE algorithm;\n- `\"all\"`: algorimths and helpers;\n- `\"parallel\"`: parallelization of the algorithms with Rayon;\n- `\"co_occurrence\"`: Co-occurrence algorithm;\n\nDefault features: `[\"tf_idf\", \"rake\", \"text_rank\"]`. By default all algorithms apart from `\"co_occurrence\"` and `\"yake\"` are enabled.\n\n\u003csmall\u003eNOTE: `\"parallel\"` feature is only recommended for large documents, it exchanges memory for computation resourses.\u003c/small\u003e\n\n### Examples\n\nFor the stop words, you can use the `stop-words` crate:\n\n```toml\n[dependencies]\nstop-words = \"0.8.0\"\n```\n\nFor example for english:\n\n```rust\nuse stop_words::{get, LANGUAGE};\n\nfn main() {\n    let stop_words = get(LANGUAGE::English);\n    let punctuation: Vec\u003cString\u003e =[\n        \".\", \",\", \":\", \";\", \"!\", \"?\", \"(\", \")\", \"[\", \"]\", \"{\", \"}\", \"\\\"\", \"'\",\n    ].iter().map(|s| s.to_string()).collect();\n    // ...\n}\n```\n\n#### TF-IDF\n\nCreate a `TfIdfParams` enum which can be one of the following:\n\n1. Unprocessed Documents: `TfIdfParams::UnprocessedDocuments`;\n2. Processed Documents: `TfIdfParams::ProcessedDocuments`;\n3. Single Unprocessed Document/Text block: `TfIdfParams::TextBlock`;\n\n```rust\nuse keyword_extraction::tf_idf::{TfIdf, TfIdfParams};\n\nfn main() {\n    // ... stop_words \u0026 punctuation\n    let documents: Vec\u003cString\u003e = vec![\n        \"This is a test document.\".to_string(),\n        \"This is another test document.\".to_string(),\n        \"This is a third test document.\".to_string(),\n    ];\n\n    let params = TfIdfParams::UnprocessedDocuments(\u0026documents, \u0026stop_words, Some(\u0026punctuation));\n\n    let tf_idf = TfIdf::new(params);\n    let ranked_keywords: Vec\u003cString\u003e = tf_idf.get_ranked_words(10);\n    let ranked_keywords_scores: Vec\u003c(String, f32)\u003e = tf_idf.get_ranked_word_scores(10);\n\n    // ...\n}\n```\n\n#### RAKE\n\nCreate a `RakeParams` enum which can be one of the following:\n\n1. With defaults: `RakeParams::WithDefaults`;\n2. With defaults and phrase length (phrase window size limit): `RakeParams::WithDefaultsAndPhraseLength`;\n3. All: `RakeParams::All`;\n\n```rust\nuse keyword_extraction::rake::{Rake, RakeParams};\n\nfn main() {\n    // ... stop_words\n    let text = r#\"\n        This is a test document.\n        This is another test document.\n        This is a third test document.\n    \"#;\n\n    let rake = Rake::new(RakeParams::WithDefaults(text, \u0026stop_words));\n    let ranked_keywords: Vec\u003cString\u003e = rake.get_ranked_words(10);\n    let ranked_keywords_scores: Vec\u003c(String, f32)\u003e = rake.get_ranked_word_scores(10);\n\n    // ...\n}\n```\n\n#### TextRank\n\nCreate a `TextRankParams` enum which can be one of the following:\n\n1. With defaults: `TextRankParams::WithDefaults`;\n2. With defaults and phrase length (phrase window size limit): `TextRankParams::WithDefaultsAndPhraseLength`;\n3. All: `TextRankParams::All`;\n\n```rust\nuse keyword_extraction::text_rank::{TextRank, TextRankParams};\n\nfn main() {\n    // ... stop_words\n    let text = r#\"\n        This is a test document.\n        This is another test document.\n        This is a third test document.\n    \"#;\n\n    let text_rank = TextRank::new(TextRankParams::WithDefaults(text, \u0026stop_words));\n    let ranked_keywords: Vec\u003cString\u003e = text_rank.get_ranked_words(10);\n    let ranked_keywords_scores: Vec\u003c(String, f32)\u003e = text_rank.get_ranked_word_scores(10);\n}\n```\n\n#### YAKE\n\nCreate a `YakeParams` enum which can be one of the following:\n\n1. With defaults: `YakeParams::WithDefaults`;\n2. All: `YakeParams::All`;\n\n```rust\nuse keyword_extraction::yake::{Yake, YakeParams};\n\nfn main() {\n    // ... stop_words\n    let text = r#\"\n        This is a test document.\n        This is another test document.\n        This is a third test document.\n    \"#;\n\n    let yake = Yake::new(YakeParams::WithDefaults(text, \u0026stop_words));\n    let ranked_keywords: Vec\u003cString\u003e = yake.get_ranked_keywords(10);\n    let ranked_keywords_scores: Vec\u003c(String, f32)\u003e = yake.get_ranked_keyword_scores(10);\n    // ...\n}\n```\n\n## Contributing\n\nI would love your input! I want to make contributing to this project as easy and transparent as possible, please read the [CONTRIBUTING.md](CONTRIBUTING.md) file for details.\n\n## License\n\nThis project is licensed under the GNU Lesser General Public License v3.0. See the [Copying](COPYING)\nand [Copying Lesser](COPYING.LESSER) files for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftugascript%2Fkeyword-extraction-rs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftugascript%2Fkeyword-extraction-rs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftugascript%2Fkeyword-extraction-rs/lists"}