Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/dmlc/gluon-nlp
NLP made easy
https://github.com/dmlc/gluon-nlp
deep-learning gluon gluonnlp machine-learning mxnet natural-language-generation natural-language-inference natural-language-processing natural-language-understanding nlg nlp nlu numpy
Last synced: 4 months ago
JSON representation
NLP made easy
- Host: GitHub
- URL: https://github.com/dmlc/gluon-nlp
- Owner: dmlc
- License: apache-2.0
- Archived: true
- Created: 2018-04-04T20:57:13.000Z (almost 7 years ago)
- Default Branch: master
- Last Pushed: 2023-10-06T04:01:21.000Z (over 1 year ago)
- Last Synced: 2024-09-21T17:45:35.139Z (4 months ago)
- Topics: deep-learning, gluon, gluonnlp, machine-learning, mxnet, natural-language-generation, natural-language-inference, natural-language-processing, natural-language-understanding, nlg, nlp, nlu, numpy
- Language: Python
- Homepage: https://nlp.gluon.ai/
- Size: 10.9 MB
- Stars: 2,555
- Watchers: 95
- Forks: 538
- Open Issues: 272
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Codeowners: CODEOWNERS
Awesome Lists containing this project
- awesome-bert - dmlc/gluon-nlp
- awesome-python-machine-learning-resources - GitHub - 44% open · ⏱️ 24.08.2021): (文本数据和NLP)
- Awesome-MXNet - **gluon-nlp**(Gluon:star:)
- awesome-nlp-note - GluonNLP - A deep learning toolkit for NLP, built on MXNet/Gluon, for research prototyping and industrial deployment of state-of-the-art models on a wide range of NLP tasks. (Libraries / Videos and Online Courses)
- awesome-production-machine-learning - GluonNLP - nlp.svg?style=social) - GluonNLP is a toolkit that enables easy text preprocessing, datasets loading and neural models building to help you speed up your Natural Language Processing (NLP) research. (Industrial Strength NLP)
- Awesome-AIML-Data-Ops - GluonNLP - nlp.svg?style=social) - GluonNLP is a toolkit that enables easy text preprocessing, datasets loading and neural models building to help you speed up your Natural Language Processing (NLP) research. (AutoML NLP)
- awesome-list - GluonNLP - A high-level NLP toolkit, based on MXNet. (Natural Language Processing / General Purpose NLP)
- awesome-production-machine-learning - GluonNLP - nlp.svg?style=social) - GluonNLP is a toolkit that enables easy text preprocessing, datasets loading and neural models building to help you speed up your Natural Language Processing (NLP) research. (Industry Strength NLP)
README
GluonNLP: Your Choice of Deep Learning for NLP
GluonNLP is a toolkit that helps you solve NLP problems. It provides easy-to-use tools that helps you load the text data,
process the text data, and train models.
See our documents at https://nlp.gluon.ai/master/index.html.
# Features
- Easy-to-use Text Processing Tools and Modular APIs
- Pretrained Model Zoo
- Write Models with Numpy-like API
- Fast Inference via [Apache TVM (incubating)](https://tvm.apache.org/) (Experimental)
- AWS Integration via [SageMaker](https://aws.amazon.com/sagemaker/)
# Installation
First of all, install the MXNet 2 release such as MXNet 2 Alpha. You may use the
following commands:
```bash
# Install the version with CUDA 10.2
python3 -m pip install -U --pre "mxnet-cu102>=2.0.0a"
# Install the version with CUDA 11
python3 -m pip install -U --pre "mxnet-cu110>=2.0.0a"
# Install the cpu-only version
python3 -m pip install -U --pre "mxnet>=2.0.0a"
```
To install GluonNLP, use
```bash
python3 -m pip install -U -e .
# Also, you may install all the extra requirements via
python3 -m pip install -U -e ."[extras]"
```
If you find that you do not have the permission, you can also install to the user folder:
```bash
python3 -m pip install -U -e . --user
```
For Windows users, we recommend to use the [Windows Subsystem for Linux](https://docs.microsoft.com/en-us/windows/wsl/about).
# Access the Command-line Toolkits
To facilitate both the engineers and researchers, we provide command-line-toolkits for
downloading and processing the NLP datasets. For more details, you may refer to
[GluonNLP Datasets](./scripts/datasets) and [GluonNLP Data Processing Tools](./scripts/processing).
```bash
# CLI for downloading / preparing the dataset
nlp_data help
# CLI for accessing some common data processing scripts
nlp_process help
# Also, you can use `python -m` to access the toolkits
python3 -m gluonnlp.cli.data help
python3 -m gluonnlp.cli.process help
```
# Run Unittests
You may go to [tests](tests) to see how to run the unittests.
# Use Docker
You can use Docker to launch a JupyterLab development environment with GluonNLP installed.
```
# GPU Instance
docker pull gluonai/gluon-nlp:gpu-latest
docker run --gpus all --rm -it -p 8888:8888 -p 8787:8787 -p 8786:8786 --shm-size=2g gluonai/gluon-nlp:gpu-latest
# CPU Instance
docker pull gluonai/gluon-nlp:cpu-latest
docker run --rm -it -p 8888:8888 -p 8787:8787 -p 8786:8786 --shm-size=2g gluonai/gluon-nlp:cpu-latest
```
For more details, you can refer to the guidance in [tools/docker](tools/docker).