{"id":16135212,"url":"https://github.com/k4black/fast-aug","last_synced_at":"2026-03-07T00:32:35.941Z","repository":{"id":224032530,"uuid":"732604054","full_name":"k4black/fast-aug","owner":"k4black","description":"Fast Augmentation library for NLP","archived":false,"fork":false,"pushed_at":"2025-04-28T15:03:52.000Z","size":829,"stargazers_count":1,"open_issues_count":9,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-16T05:36:20.956Z","etag":null,"topics":["augmentation","natural-language-processing","nlp","python","rust","text-augmentation"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/k4black.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-12-17T08:30:50.000Z","updated_at":"2025-01-27T13:26:48.000Z","dependencies_parsed_at":"2024-03-22T10:50:01.357Z","dependency_job_id":"dc98cdf7-ffcf-4573-89a1-c1eb7027b8da","html_url":"https://github.com/k4black/fast-aug","commit_stats":{"total_commits":102,"total_committers":3,"mean_commits":34.0,"dds":0.1568627450980392,"last_synced_commit":"c4a8fe979e43cf2c5262b7bfaf783311efa8a80b"},"previous_names":["k4black/fast-aug"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/k4black/fast-aug","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k4black%2Ffast-aug","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k4black%2Ffast-aug/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k4black%2Ffast-aug/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k4black%2Ffast-aug/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/k4black","download_url":"https://codeload.github.com/k4black/fast-aug/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k4black%2Ffast-aug/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30204154,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-06T19:07:06.838Z","status":"ssl_error","status_checked_at":"2026-03-06T18:57:34.882Z","response_time":250,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["augmentation","natural-language-processing","nlp","python","rust","text-augmentation"],"created_at":"2024-10-09T23:06:12.954Z","updated_at":"2026-03-07T00:32:35.902Z","avatar_url":"https://github.com/k4black.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# fast-aug\n\n[![Rust Test Workflow Status](https://img.shields.io/github/actions/workflow/status/k4black/fast-aug/test-rust.yml?branch=main\u0026event=push\u0026label=rust%20tests)](https://github.com/k4black/fast-aug/actions/workflows/test-rust.yml)\n[![Python Test Workflow Status](https://img.shields.io/github/actions/workflow/status/k4black/fast-aug/test-python.yml?branch=main\u0026event=push\u0026label=python%20tests)](https://github.com/k4black/fast-aug/actions/workflows/test-python.yml)\n\n[![Crates.io Version](https://img.shields.io/crates/v/fast-aug)](https://crates.io/crates/fast-aug)\n[![PyPI - Version](https://img.shields.io/pypi/v/fast-aug)](https://pypi.org/project/fast-aug/)\n[![GitHub License](https://img.shields.io/github/license/k4black/fast-aug)](https://github.com/k4black/fast-aug/blob/main/LICENSE)\n\n\n`fast-aug` is a library for fast text augmentation, available for both Rust and Python as `fast-aug`.  \nIt is designed with focus on performance and real-time usage (e.g. during training), while providing a wide range of text augmentation methods.\n\n---\n\nPlease refer to respective READMEs for details:\n* `rust` - see [fast_aug/README.md](fast_aug/README.md);\n* `python` - see [bindings/python/README.md](bindings/python/README.md).\n\n\n## Features and TODO\n\nFlow\n- [x] ChanceAugmenter\n- [x] SelectorAugmenter\n- [x] SequentialAugmenter\n\nText\n- [ ] RandomWordsAugmenter\n  - [x] Base - swaps/deletions\n  - [x] Insertions/Substitutions (from alphabet)\n- [ ] RandomCharsAugmenter\n    - [x] Base - swaps/deletions\n    - [x] Insertions/Substitutions (from provided list)\n    - [ ] Insertions/Substitutions (from vocab by language tag)\n- [ ] RandomSpellingAugmenter\n- [ ] RandomKeyboardAugmenter\n- [ ] RandomEmbeddingsAugmenter\n- [ ] RandomTfIdfAugmenter\n- [ ] RandomPosAugmenter\n- [ ] [EmojiNormalizer](https://github.com/unicode-org/cldr-json/blob/858baad63c1d51e1d576ef99dccc229d92cedda4/cldr-json/cldr-annotations-full/annotations/en-AU/annotations.json#L1498)\n- [ ] Keep labels (e.g. POS tags) unchanged\n\nModels and utils\n- [ ] Models lazy loading\n  - [ ] At creation time\n  - [ ] At first use\n  - [ ] Background after creation\n- [ ] [candle](https://github.com/huggingface/candle) support for DL models loading\n  - [ ] HF loading\n  - [ ] ONNX loading\n  - [ ] Optimizations (fp16/int8/int4/layers/etc)\n  - [ ] GPU support\n- [ ] TF-IDF model\n  - [ ] json file loading\n  - [ ] sklearn model loading\n- [x] Alphabet model\n- [ ] Language Vocab model\n- [ ] Embeddings model\n  - [ ] fasttext model loading\n  - [ ] word2vec model loading\n- [ ] WordNet model\n  - [ ] English\n  - [ ] German\n  - [ ] More?\n\nRust\n- [x] Formatting\n  - [x] rustfmt\n  - [x] clippy\n- [x] [rust flamegraph profiling](https://www.jibbow.com/posts/criterion-flamegraphs/)\n- [x] Unit tests\n- [ ] Integration tests\n- [x] CI build and tests\n- [ ] CI publish to crates.io\n\n\nPython \n- [ ] Custom Python Augmenter class (user provided to use in pipelines)\n- [ ] Bindings with \n  - [x] Base pyo3 bindings\n  - [x] [maturin](https://github.com/PyO3/maturin) auto build from pyproject.toml\n  - [x] Stubs (.pyi) files generation\n  - [ ] Auto generate stubs on maturing build\n  - [x] Text\n  - [x] Flow\n- [ ] Auto generate return type in stubs, see [pyo3 issue](https://github.com/PyO3/pyo3/issues/1112) \n- [x] [flamegraph profiling](https://ohadravid.github.io/posts/2023-03-rusty-python/)\n- [ ] Optimizations - see [this](https://ohadravid.github.io/posts/2023-03-rusty-python/)\n- [ ] Integration tests\n- [x] CI build and tests\n- [ ] CI publish to pypi\n\n\n## Development\n\n### Prerequisites\n\nClone the repository:\n```shell\ngit clone git@github.com:k4black/fast-aug.git\ncd fast-aug\n```\n\nFor rust library development:\n* [rustup](https://rustup.rs/)\n* [cargo](https://doc.rust-lang.org/cargo/getting-started/installation.html)\n\nFor python bindings development:\n* All rust library prerequisites\n* `cd bindings/python \u0026\u0026 python -m venv .venv`\n* `pip \u003e= 23.1` to use `--config-settings`, see [pip issue](https://github.com/pypa/pip/issues/11859)\n\n### Make\n\nThe `Makefile` contains all the commands needed for development.\n```shell\nmake help\n```\n\n- `*-rust` - all targets related to rust library (`fast_aug/` folder)\n- `*-python` - all targets related to python bindings (`bindings/python/` folder)\n\n\n### Benchmarks\n\nAll text benchmarks are run on the [tweet_eval dataset](https://github.com/cardiffnlp/tweeteval/) - sentiment task, test set, 12k rows.\n```shell\ncat test_data/tweet_eval_sentiment_test_text.txt | wc\n12284  182576 1156877\n```\n\n\n## License\n\nThis project and respective libraries are licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fk4black%2Ffast-aug","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fk4black%2Ffast-aug","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fk4black%2Ffast-aug/lists"}