{"id":19031438,"url":"https://github.com/plandes/deepnlp","last_synced_at":"2026-05-03T07:30:19.021Z","repository":{"id":57478146,"uuid":"262951073","full_name":"plandes/deepnlp","owner":"plandes","description":"Deep learning utility library for natural language processing","archived":false,"fork":false,"pushed_at":"2024-03-07T17:14:13.000Z","size":7118,"stargazers_count":10,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-04-09T07:45:18.624Z","etag":null,"topics":["deep-learning","deep-neural-networks","framework","natural-language-processing","nlp"],"latest_commit_sha":null,"homepage":"https://plandes.github.io/deepnlp/","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/plandes.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2020-05-11T05:53:05.000Z","updated_at":"2024-04-14T20:31:37.326Z","dependencies_parsed_at":"2024-02-27T18:29:33.029Z","dependency_job_id":"5b70114a-ca02-46ba-be97-3a58a8a84fe3","html_url":"https://github.com/plandes/deepnlp","commit_stats":null,"previous_names":[],"tags_count":30,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plandes%2Fdeepnlp","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plandes%2Fdeepnlp/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plandes%2Fdeepnlp/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/plandes%2Fdeepnlp/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/plandes","download_url":"https://codeload.github.com/plandes/deepnlp/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240080758,"owners_count":19744920,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","deep-neural-networks","framework","natural-language-processing","nlp"],"created_at":"2024-11-08T21:23:27.603Z","updated_at":"2026-05-03T07:30:18.986Z","avatar_url":"https://github.com/plandes.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# DeepZensols Natural Language Processing\n\n[![PyPI][pypi-badge]][pypi-link]\n[![Python 3.11][python311-badge]][python311-link]\n[![Build Status][build-badge]][build-link]\n\nDeep learning utility library for natural language processing that aids in\nfeature engineering and embedding layers.\n\n* See the [full documentation].\n* See the [paper](https://aclanthology.org/2023.nlposs-1.16)\n\nFeatures:\n* Configurable layers with little to no need to write code.\n* [Natural language specific layers]:\n  * Easily configurable word embedding layers for [Glove], [Word2Vec],\n    [fastText].\n  * Huggingface transformer ([BERT]) context based word vector layer.\n  * Full [Embedding+BiLSTM-CRF] implementation using easy to configure\n\tconstituent layers.\n* [NLP specific vectorizers] that generate [zensols deeplearn] encoded and\n  decoded [batched tensors] for [spaCy] parsed features, dependency tree\n  features, overlapping text features and others.\n* Easily swapable during runtime embedded layers as [batched tensors] and other\n  linguistic vectorized features.\n* Support for token, document and embedding level vectorized features.\n* Transformer word piece to linguistic token mapping.\n* Two full documented reference models provided as both command line and\n  [Jupyter notebooks](#usage-and-reference-models).\n* Command line support for training, testing, debugging, and creating\n  predictions.\n\n\n## Documentation\n\n* [Full documentation](https://plandes.github.io/deepnlp/index.html)\n* [Layers](https://plandes.github.io/deepnlp/doc/layers.html): NLP specific\n  layers such as embeddings and transformers\n* [Vectorizers](https://plandes.github.io/deepnlp/doc/vectorizers.html):\n  specific vectorizers that digitize natural language text in to tensors ready\n  as [PyTorch] input\n* [API reference](https://plandes.github.io/install/api.html)\n* [Reference Models](#usage-and-reference-models)\n\n\n## Obtaining\n\nThe easiest way to install the command line program is via the `pip` installer:\n```bash\npip3 install zensols.deepnlp\n```\n\nBinaries are also available on [pypi].\n\n\n## Usage\n\nThe API can be used as is and manually configuring each component.  However,\nthis (like any Zensols API) was designed to instantiated with inverse of\ncontrol using [resource libraries].\n\n### Component\n\nComponents and out of the box models are available with little to no coding.\nHowever, this [simple example](example/simple/harness.py) that uses the\nlibrary's components is recommended for starters.  The example is a command\nline application that in-lines a simple configuration needed to create deep\nlearning NLP components.\n\nSimilarly, [this example](example/fill-mask/harness.py) is also a command line\nexample, but uses a masked langauge model to fill in words.\n\n\n### Reference Models\n\nIf you're in a rush, you can dive right in to the [Clickbate Text\nClassification] reference model, which is a working project that uses this\nlibrary.  However, you'll either end up reading up on the [zensols deeplearn]\nlibrary before or during the tutorial.\n\nThe usage of this library is explained in terms of the reference models:\n\n* The [Clickbate Text Classification] is the best reference model to start with\n  because the only code consists of is the corpus reader and a module to remove\n  sentence segmentation (corpus are newline delimited headlines).  It was also\n  uses [resource libraries], which greatly reduces complexity, where as the\n  other reference models do not.  Also see the [Jupyter clickbate\n  classification notebook].\n\n* The [Movie Review Sentiment] trained and tested on the [Stanford movie\n  review] and [Cornell sentiment polarity] data sets, which assigns a positive\n  or negative score to a natural language movie review by critics.  Also see\n  the [Jupyter movie sentiment notebook].\n\n* The [Named Entity Recognizer] trained and tested on the [CoNLL 2003 data set]\n  to label named entities on natural language text.  Also see the [Jupyter NER\n  notebook].\n\nThe unit test cases are also a good resource for the more detailed programming\nintegration with various parts of the library.\n\n\n## Attribution\n\nThis project, or reference model code, uses:\n* [Gensim] for [Glove], [Word2Vec] and [fastText] word embeddings.\n* [Huggingface Transformers] for [BERT] contextual word embeddings.\n* [h5py] for fast read access to word embedding vectors.\n* [zensols nlparse] for feature generation from [spaCy] parsing.\n* [zensols deeplearn] for deep learning network libraries.\n\nCorpora used include:\n* [Stanford movie review]\n* [Cornell sentiment polarity]\n* [CoNLL 2003 data set]\n\n\n## Citation\n\nIf you use this project in your research please use the following BibTeX entry:\n\n```bibtex\n@inproceedings{landes-etal-2023-deepzensols,\n    title = \"{D}eep{Z}ensols: A Deep Learning Natural Language Processing Framework for Experimentation and Reproducibility\",\n    author = \"Landes, Paul  and\n      Di Eugenio, Barbara  and\n      Caragea, Cornelia\",\n    editor = \"Tan, Liling  and\n      Milajevs, Dmitrijs  and\n      Chauhan, Geeticka  and\n      Gwinnup, Jeremy  and\n      Rippeth, Elijah\",\n    booktitle = \"Proceedings of the 3rd Workshop for Natural Language Processing Open Source Software (NLP-OSS 2023)\",\n    month = dec,\n    year = \"2023\",\n    address = \"Singapore, Singapore\",\n    publisher = \"Association for Computational Linguistics\",\n    url = \"https://aclanthology.org/2023.nlposs-1.16\",\n    pages = \"141--146\"\n}\n```\n\n\n## Changelog\n\nAn extensive changelog is available [here](CHANGELOG.md).\n\n\n## Community\n\nPlease star this repository and let me know how and where you use this API.\nContributions as pull requests, feedback and any input is welcome.\n\n\n## License\n\n[MIT License](LICENSE.md)\n\nCopyright (c) 2020 - 2025 Paul Landes\n\n\n\u003c!-- links --\u003e\n[pypi]: https://pypi.org/project/zensols.deepnlp/\n[pypi-link]: https://pypi.python.org/pypi/zensols.deepnlp\n[pypi-badge]: https://img.shields.io/pypi/v/zensols.deepnlp.svg\n[python311-badge]: https://img.shields.io/badge/python-3.11-blue.svg\n[python311-link]: https://www.python.org/downloads/release/python-3110\n[build-badge]: https://github.com/plandes/util/workflows/CI/badge.svg\n[build-link]: https://github.com/plandes/deepnlp/actions\n\n[PyTorch]: https://pytorch.org\n[Gensim]: https://radimrehurek.com/gensim/\n[Huggingface Transformers]: https://huggingface.co\n[Glove]: https://nlp.stanford.edu/projects/glove/\n[Word2Vec]: https://code.google.com/archive/p/word2vec/\n[fastText]: https://fasttext.cc\n[BERT]: https://huggingface.co/transformers/model_doc/bert.html\n[h5py]: https://www.h5py.org\n[spaCy]: https://spacy.io\n[Pandas]: https://pandas.pydata.org\n\n[Stanford movie review]: https://nlp.stanford.edu/sentiment/\n[Cornell sentiment polarity]: https://www.cs.cornell.edu/people/pabo/movie-review-data/\n[CoNLL 2003 data set]: https://www.clips.uantwerpen.be/conll2003/ner/\n\n[zensols deeplearn]: https://github.com/plandes/deeplearn\n[zensols nlparse]: https://github.com/plandes/nlparse\n\n[full documentation]: https://plandes.github.io/deepnlp/index.html\n[resource libraries]: https://plandes.github.io/util/doc/config.html#resource-libraries\n[Natural language specific layers]: https://plandes.github.io/deepnlp/doc/layers.html\n[Clickbate Text Classification]: https://plandes.github.io/deepnlp/doc/clickbate-example.html\n[Movie Review Sentiment]: https://plandes.github.io/deepnlp/doc/movie-example.html\n[Named Entity Recognizer]: https://plandes.github.io/deepnlp/doc/ner-example.html\n[Embedding+BiLSTM-CRF]: https://plandes.github.io/deepnlp/doc/ner-example.html#bilstm-crf\n[batched tensors]: https://plandes.github.io/deeplearn/doc/preprocess.html#batches\n[deep convolution layer]: https://plandes.github.io/deepnlp/api/zensols.deepnlp.layer.html#zensols.deepnlp.layer.conv.DeepConvolution1d\n[NLP specific vectorizers]: https://plandes.github.io/deepnlp/doc/vectorizers.html\n[Jupyter NER notebook]: https://github.com/plandes/deepnlp/blob/master/example/ner/notebook/ner.ipynb\n[Jupyter movie sentiment notebook]: https://github.com/plandes/deepnlp/blob/master/example/movie/notebook/movie.ipynb\n[Jupyter clickbate classification notebook]: https://github.com/plandes/deepnlp/blob/master/example/clickbate/notebook/clickbate.ipynb\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fplandes%2Fdeepnlp","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fplandes%2Fdeepnlp","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fplandes%2Fdeepnlp/lists"}