https://github.com/redis-developer/rediner
Named Entity Recognition for Redis
https://github.com/redis-developer/rediner
flask graph named-entity-recognition nlp property-graph redis redisgraph serverless spacy-nlp
Last synced: about 1 year ago
JSON representation
Named Entity Recognition for Redis
- Host: GitHub
- URL: https://github.com/redis-developer/rediner
- Owner: redis-developer
- License: mit
- Created: 2020-10-19T17:59:28.000Z (over 5 years ago)
- Default Branch: main
- Last Pushed: 2020-10-29T19:33:06.000Z (over 5 years ago)
- Last Synced: 2025-01-25T09:27:37.990Z (about 1 year ago)
- Topics: flask, graph, named-entity-recognition, nlp, property-graph, redis, redisgraph, serverless, spacy-nlp
- Language: Python
- Homepage: https://redis-developer.github.io/rediner/
- Size: 119 KB
- Stars: 2
- Watchers: 3
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# rediner
[Named Entity Recognition](https://en.wikipedia.org/wiki/Named-entity_recognition) (NER) analysis in RedisGraph.
# Overview
This project contains an example of using the [SpaCy](https://spacy.io)
NER model to harvest
named entities from blog posts or other web pages. The crawler program
produces graph structures that can be loaded into RedisGraph for analysis
via Cypher Queries.
# Demo
A demo is available [online](http://rediner.milowski.io). Please be kind!
# A Quick Start
1. Create an environment an load the requirements:
```
pip install -r requirements.txt
python -m spacy download en_core_web_sm
```
1. Collect some entries:
```
mkdir out
python -m rediner.blog --same --store --dir out --verbose https://www.milowski.com/
```
1. Run the SpaCy model:
```
python -m rediner --yaml -r ner out > milowski-com-terms.yaml
```
1. Start RedisGraph:
```
docker run -p 6379:6379 redislabs/redisgraph:latest
```
1. Setup the indexes:
```
python demo/setupdb.py milowski.com
```
1. Ingest the data:
```
python -m rediner load -r --graph milowski.com out
python -m rediner load -r --graph milowski.com milowski-com-terms.yaml
```
1. Run the demo application:
```
cd demo
python view.py
```
1. View the application locally at http://localhost:5000/
Once the application is running, you can examine the dataset in various ways. For starters, try loading the graphs with the defaults by clicking on the 'Load'
button. This will load the entities that match the minimum counts and show a
graph of cooccurrences.
You can:
* use the "same" checkbox to enable entities that co-occur on the same article.
* use the "multiple" checkbox to enable entities that co-occur on more than one article.
* use search to access the full text search. This will display a subgraph of
entities from the search results.
* clicking on a node in the graph will show the articles which contain the
entities with a link back to the original resource.
# Next steps
You can view more extensive document on the [website](https://redis-developer.github.io/rediner/).