Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/fastent/fastent
custom models for named-entity recognition
https://github.com/fastent/fastent
data-annotation data-generation named-entities named-entity-recognition natural-language-processing nlp spacy
Last synced: 2 months ago
JSON representation
custom models for named-entity recognition
- Host: GitHub
- URL: https://github.com/fastent/fastent
- Owner: fastent
- License: mit
- Created: 2018-01-23T15:50:40.000Z (almost 7 years ago)
- Default Branch: master
- Last Pushed: 2021-03-31T18:35:23.000Z (over 3 years ago)
- Last Synced: 2024-09-29T22:05:03.006Z (3 months ago)
- Topics: data-annotation, data-generation, named-entities, named-entity-recognition, natural-language-processing, nlp, spacy
- Language: Python
- Homepage: https://fastent.github.io
- Size: 2.58 MB
- Stars: 7
- Watchers: 3
- Forks: 2
- Open Issues: 10
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# fastent
The **fastent** Python library is a tool for end-to-end creation of **custom models for [named-entity recognition](https://en.wikipedia.org/wiki/Named-entity_recognition)**.
#### Custom Models
To train a model for a new type of entity, you just need a list of examples.
You are not limited to only predefined types like person, location and organization.
#### How It Works
fastent does end-to-end creation: **dataset generation**, **annotation**, **contextualiziation** and **training** a model.
You can also use fastent modules as standalone tools.
#### Made for Prod
fastent includes integrations with tools like spaCy, fastText pre-trained models and NLTK.
fastent is built to scale to very large text datasets in many languages.
---
* [Installation](#installation)
* [How To](#how-to)
* [Dataset Generation](#generation)
* [Annotation](#annotation)
* [Contextualization](#contextualization)
* [Training](#training)
* [Testing](#testing)
* [Integrations](#integrations)
* [Pre-trained Models](#pre-trained-models)
* [Text utilities](#text-utilities)
* [WordNet](#wordnet)
* [Poincaré embeddings](#poincare-embeddings)
* [More](#more)
### Installation
fastent is developed for Python 3 on Unix systems.
Clone this repo or install from PyPI:
```
pip install fastent
```
Download NLTK data:
```
python -m nltk.downloader stopwords
```
Install and set up CouchDB:
```
wget -O - https://raw.githubusercontent.com/fastent/fastent/master/install.sh | bash
```
#### Downloading data files
TODO: fastText stuff
## How To
### Generation
fastent can generate a dataset from a list
TODO
fastent can even generate a list from one or two examples.
```
from fastent import dataset_pseudo_generator
model = dataset_pseudo_generator.spacy_initialize('en_core_web_lg')
dataset_pseudo_generator.dataset_generate(model,['cocaine', 'heroin'], 100)
```
The equivalent on the command line:
```
python dataset_pseudo_generator.py -m en_core_web_lg -s cocaine,heroin
```
### Annotation
TODO
### Contextualization
TODO
### Training
To train a model from the annotated and contextualized dataset:
For now the only supported learning framework is spaCy.
[Request support for a new learning framework](https://github.com/fastent/fastent/issues/new?labels=Models&title=New+learning+framework+support+request:)
TODO: sample output
### Testing
Coming soon!
## Integrations
fastent includes integrations for downloading datasets and pre-trained models.
TODO
## More
See how fastent performs on [benchmarks](/benchmarks)
Try the [tutorial](/tutorial) or fork [examples](/examples)
Browse [frequently asked questions](/faq)
[Report bugs or request new features](https://github.com/fastent/fastent/issues/new)