https://github.com/jedirhymetrix/profanityfilter
Simple profanity filter in python
https://github.com/jedirhymetrix/profanityfilter
machine-learning nlp profanity-filter python rest-api sentiment-analysis wsd
Last synced: 3 months ago
JSON representation
Simple profanity filter in python
- Host: GitHub
- URL: https://github.com/jedirhymetrix/profanityfilter
- Owner: JediRhymeTrix
- Created: 2018-01-14T13:17:16.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2021-05-18T09:23:45.000Z (about 4 years ago)
- Last Synced: 2025-01-04T00:47:38.176Z (5 months ago)
- Topics: machine-learning, nlp, profanity-filter, python, rest-api, sentiment-analysis, wsd
- Language: OpenEdge ABL
- Size: 21.8 MB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# ProfanityFilter
Simple profanity filter and sentiment classifier in python
NOTE: To add words to the database, add the newline separated words to `data/feed_list.txt` and run `FeedList.py` with 2-letter language code as command line
argument. \
eg. `python FeedList.py -l en`
## Instructions:
-
Docker:
- Build: `docker-compose build`
- Run: `docker-compose up`
- ### API:
- URL: [http://localhost:5000/](http://localhost:5000/)
- Endpoints:
- **/process** ```["POST"]```
* *request* - JSON in the following format:
````json
{
"post": ,
"options": ["filter", "sentiment" / "sentiment_heavy"],
"wordlist_url": ,
"ignore_words_url":
}
```
* *response* - JSON in the following format:
if "sentiment" in request:
```json
{
"profanities": [],
"sentiment":
{
"confidence": "med" / "high" / "low",
"polarity": -1 to +1 (-ve to +ve)
}
}
```
if "sentiment_heavy" in request:
```json
{
"profanities": [],
"sentiment":
{
"polarity": -1 to +1 (-ve to +ve)
}
}
```
**NOTE**: ```sentiment_heavy``` is *slow* but more accurate. use ```sentiment``` first and then ```sentiment_heavy``` can be used based on the confidence level returned.
Meanings of *confidence levels*:
- high: **high polarity** and **low subjectivity**
- med: **high polarity** and **high subjectivity** (classification is mostly correct but exact score may be inaccurate) - ```sentiment_heavy``` might give a better score.
- low: **low polarity** and **high subjectivity** - low accuracy possible. Use ```sentiment_heavy``` instead.
## Examples:
1.
```json
req: {
"post": "Well, I completely disagree with this “wonderful” lady who doesn’t know anything and acts like she has never set foot in Spain. God, I’d like to gag her! fuck chutiya",
"options": ["filter"],
}
res: {
"profanities": [
"fuck",
"chutiya"
],
}
```
2.
```json
req: {
"post": "Well, I completely disagree with this “wonderful” lady who doesn’t know anything and acts like she has never set foot in Spain. God, I’d like to gag her! fuck chutiya",
"options": ["filter"],
"url": "https://raw.githubusercontent.com/LDNOOBW/List-of-Dirty-Naughty-Obscene-and-Otherwise-Bad-Words/master/en"
}
res: {
"profanities": [
"fuck",
"chutiya"
],
}
```
3.
```json
req: {
"post": "Well, I completely disagree with this “wonderful” lady who doesn’t know anything and acts like she has never set foot in Spain. God, I’d like to gag her! fuck chutiya",
"options": ["filter", "sentiment"],
"url": "https://raw.githubusercontent.com/LDNOOBW/List-of-Dirty-Naughty-Obscene-and-Otherwise-Bad-Words/master/en"
}
res: {
"profanities": [
"fuck",
"chutiya"
],
"sentiment": {
"confidence": "low",
"polarity": 0.2
}
}
```
4.
```json
req: {
"post": "Well, I completely disagree with this “wonderful” lady who doesn’t know anything and acts like she has never set foot in Spain. God, I’d like to gag her! fuck chutiya",
"options": ["filter", "sentiment_heavy"],
"url": "https://raw.githubusercontent.com/LDNOOBW/List-of-Dirty-Naughty-Obscene-and-Otherwise-Bad-Words/master/en"
}
res: {
"profanities": [
"fuck",
"chutiya"
],
"sentiment": {
"polarity": -0.8
}
}
```