{"id":24500752,"url":"https://github.com/inseefrlab/torch-fasttext","last_synced_at":"2025-07-06T19:05:49.754Z","repository":{"id":219111523,"uuid":"747237560","full_name":"InseeFrLab/torch-fastText","owner":"InseeFrLab","description":"Efficient text classification with Pytorch","archived":false,"fork":false,"pushed_at":"2025-04-08T09:18:40.000Z","size":7587,"stargazers_count":13,"open_issues_count":10,"forks_count":4,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-08T10:21:20.278Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/InseeFrLab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-23T14:42:37.000Z","updated_at":"2025-04-08T09:18:45.000Z","dependencies_parsed_at":"2024-12-12T18:28:27.700Z","dependency_job_id":"66683ab0-7958-443d-9c95-ee67fc78f687","html_url":"https://github.com/InseeFrLab/torch-fastText","commit_stats":null,"previous_names":["inseefrlab/torch-fasttext"],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InseeFrLab%2Ftorch-fastText","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InseeFrLab%2Ftorch-fastText/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InseeFrLab%2Ftorch-fastText/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/InseeFrLab%2Ftorch-fastText/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/InseeFrLab","download_url":"https://codeload.github.com/InseeFrLab/torch-fastText/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248830420,"owners_count":21168272,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-21T22:21:48.528Z","updated_at":"2025-04-14T05:42:35.764Z","avatar_url":"https://github.com/InseeFrLab.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# torchFastText : Efficient text classification with PyTorch\n\nA flexible PyTorch implementation of FastText for text classification with support for categorical features.\n\n## Features\n\n- Supports text classification with FastText architecture\n- Handles both text and categorical features\n- N-gram tokenization\n- Flexible optimizer and scheduler options\n- GPU and CPU support\n- Model checkpointing and early stopping\n- Prediction and model explanation capabilities\n\n## Installation\n\n```bash\npip install torchFastText\n```\n\n## Key Components\n\n- `build()`: Constructs the FastText model architecture\n- `train()`: Trains the model with built-in callbacks and logging\n- `predict()`: Generates class predictions\n- `predict_and_explain()`: Provides predictions with feature attributions\n\n## Subpackages\n\n- `preprocess`: To preprocess text input, using `nltk` and `unidecode` libraries.\n- `explainability`: Simple methods to visualize feature attributions at word and letter levels, using `captum`library.\n\nRun `pip install torchFastText[preprocess]` or `pip install torchFastText[explainability]` to download these optional dependencies.\n\n\n## Quick Start\n\n```python\nfrom torchFastText import torchFastText\n\n# Initialize the model\nmodel = torchFastText(\n    num_tokens=1000000,\n    embedding_dim=100,\n    min_count=5,\n    min_n=3,\n    max_n=6,\n    len_word_ngrams=True,\n    sparse=True\n)\n\n# Train the model\nmodel.train(\n    X_train=train_data,\n    y_train=train_labels,\n    X_val=val_data,\n    y_val=val_labels,\n    num_epochs=10,\n    batch_size=64,\n    lr=4e-3\n)\n# Make predictions\npredictions = model.predict(test_data)\n```\n\nwhere ```train_data``` is an array of size $(N,d)$, having the text in string format in the first column, the other columns containing tokenized categorical variables in `int` format.\n\nPlease make sure `y_train` contains at least one time each possible label.\n\n## Dependencies\n\n- PyTorch Lightning\n- NumPy\n\n## Categorical features\n\nIf any, each categorical feature $i$ is associated to an embedding matrix of size (number of unique values, embedding dimension) where the latter is a hyperparameter (`categorical_embedding_dims`) - chosen by the user - that can take three types of values:\n\n- `None`: same embedding dimension as the token embedding matrix. The categorical embeddings are then summed to the sentence-level embedding (which itself is an averaging of the token embeddings). See [Figure 1](#Default-architecture).\n- `int`: the categorical embeddings have all the same embedding dimensions, they are averaged and the resulting vector is concatenated to the sentence-level embedding (the last linear layer has an adapted input size). See [Figure 2](#avg-architecture).\n- `list`: the categorical embeddings have different embedding dimensions, all of them are concatenated without aggregation to the sentence-level embedding (the last linear layer has an adapted input size). See [Figure 3](#concat-architecture).\n\nDefault is `None`.\n\n\u003ca name=\"figure-1\"\u003e\u003c/a\u003e\n![Default-architecture](images/NN.drawio.png \"Default architecture\")  \n*Figure 1: The 'sum' architecture*\n\n\u003ca name=\"figure-2\"\u003e\u003c/a\u003e\n![avg-architecture](images/avg_concat.png \"Default architecture\")  \n*Figure 2: The 'average and concatenate' architecture*\n\n\u003ca name=\"figure-3\"\u003e\u003c/a\u003e\n![concat-architecture](images/full_concat.png \"Default architecture\")  \n*Figure 3: The 'concatenate all' architecture*\n\n## Documentation\n\nFor detailed usage and examples, please refer to the [example notebook](notebooks/example.ipynb). Use `pip install -r requirements.txt` after cloning the repository to install the necessary dependencies (some are specific to the notebook).\n\n## Contributing\n\nContributions are welcome! Please feel free to submit a Pull Request.\n\n## License\n\nMIT\n\n\n## References\n\nInspired by the original FastText paper [1] and implementation.\n\n[1] A. Joulin, E. Grave, P. Bojanowski, T. Mikolov, [*Bag of Tricks for Efficient Text Classification*](https://arxiv.org/abs/1607.01759)\n\n```\n@InProceedings{joulin2017bag,\n  title={Bag of Tricks for Efficient Text Classification},\n  author={Joulin, Armand and Grave, Edouard and Bojanowski, Piotr and Mikolov, Tomas},\n  booktitle={Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers},\n  month={April},\n  year={2017},\n  publisher={Association for Computational Linguistics},\n  pages={427--431},\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finseefrlab%2Ftorch-fasttext","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Finseefrlab%2Ftorch-fasttext","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finseefrlab%2Ftorch-fasttext/lists"}