Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/softmarshmallow/inked-engine

πŸ€– natural language processing out of the box
https://github.com/softmarshmallow/inked-engine

django nlp nltk python

Last synced: about 1 month ago
JSON representation

πŸ€– natural language processing out of the box

Awesome Lists containing this project

README

        

# inked engine πŸ€–πŸ€–

| λ‰΄μŠ€ 뢄석을 μœ„ν•œ νˆ΄ν‚·μž…λ‹ˆλ‹€.

Main features

* news data indexing

* news data processing

* provide api for service server

Inked-news-crawler μ—μ„œ μƒˆλ‘œμš΄ λ‰΄μŠ€λ°μ΄ν„°λ₯Ό λ°›μ•„μ˜¨ν›„, 인덱싱과 pre-proccessing 을 ν•©λ‹ˆλ‹€. μ„œλΉ„μŠ€ μ„œλ²„μ—μ„œ μš”μ²­ν•˜λŠ” 정보λ₯Ό λΆ„μ„ν•˜μ—¬ μ„œλΉ„μŠ€ μ„œλ²„λ‘œ μ „λ‹¬ν•˜λ©°, μ„œλΉ„μŠ€ μ„œλ²„μ—μ„œ ν΄λΌμ΄μ–ΈνŠΈλ‘œ λ‰΄μŠ€ 정보λ₯Ό μ œκ³΅ν•©λ‹ˆλ‹€.

## News data model

- tags : { company : [], namedEntities: [], keywords: []}
- content
- origin
- title
- time

# How to install virtualenv:

### Install **pip** first

sudo apt-get install python3-pip

### Then install **virtualenv** using pip3

sudo pip3 install virtualenv

### Now create a virtual environment

virtualenv venv

## KoNlPy setup
http://konlpy.org/en/v0.4.4/install/
`sudo apt-get install g++ openjdk-8-jdk`
`bash <(curl -s https://raw.githubusercontent.com/konlpy/konlpy/master/scripts/mecab.sh)`

## start the engine server
`daphne server.asgi:application`

## supervisor ctrl
restart server
`sudo supervisorctl restart asgi_daphne`

## IMPORTANT:: seed credential files
you can see
```gitignore
server/settings/production.py
credentials/db-connection.json
```
from `.gitignore` which two files you will have to provide manually to run this project.

## modules
- duplicate news checker βœ…
- spam news detector 🚫
- word2vec βœ… (wiki) 🚫 (news)

## used by
* [wor.io](https://github.com/softmarshmallow/wor.io)
* [inked-server](https://github.com/softmarshmallow/inked-server)

## developed by
develped by softmarshmallow