Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/mmguero/montag
Montag is a utility which reads e-book files and scrubs them of profanity
https://github.com/mmguero/montag
e-book ebook objectional-language obscenity profanity profanity-detection profanity-filter profanityfilter python python3 swear-filter
Last synced: about 1 month ago
JSON representation
Montag is a utility which reads e-book files and scrubs them of profanity
- Host: GitHub
- URL: https://github.com/mmguero/montag
- Owner: mmguero
- License: bsd-3-clause
- Created: 2018-11-09T03:46:29.000Z (about 6 years ago)
- Default Branch: main
- Last Pushed: 2024-10-22T19:57:01.000Z (2 months ago)
- Last Synced: 2024-11-15T10:59:05.748Z (about 1 month ago)
- Topics: e-book, ebook, objectional-language, obscenity, profanity, profanity-detection, profanity-filter, profanityfilter, python, python3, swear-filter
- Language: Python
- Homepage:
- Size: 73.2 KB
- Stars: 7
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Montag
[](https://pypi.python.org/pypi/montag-cleaner/) [](https://github.com/mmguero/montag/pkgs/container/montag)
*"Didn't firemen prevent fires rather than stoke them up and get them going?"*
**montag** is a utility which reads an e-book file (in any format supported by [Calibre's ebook-convert](https://manual.calibre-ebook.com/generated/en/ebook-convert.html)) and scrubs it of profanity (or words from any other list you can provide).
There are all sorts of arguments to be had about obscenity filters, censorship, etc. That's okay! I'm not really interested in having those arguments. My 13 year-old daughter asked me if I could take some swear words out of a young adult novel she was reading so I wrote this for her. If it's useful to you, great. If not, carry on my wayward son.
**montag** is part of a family of projects with similar goals:
* 📼 [cleanvid](https://github.com/mmguero/cleanvid) for video files (using [SRT-formatted](https://en.wikipedia.org/wiki/SubRip#Format) subtitles)
* 🎤 [monkeyplug](https://github.com/mmguero/monkeyplug) for audio and video files (using either [Whisper](https://openai.com/research/whisper) or the [Vosk](https://alphacephei.com/vosk/)-[API](https://github.com/alphacep/vosk-api) for speech recognition)
* 📕 [montag](https://github.com/mmguero/montag) for ebooks
## Installation
Using `pip`, to install the latest [release from PyPI](https://pypi.org/project/montag-cleaner/):
```
python3 -m pip install -U montag-cleaner
```
Or to install directly from GitHub:
```
python3 -m pip install -U 'git+https://github.com/mmguero/montag'
```
## Prerequisites
### Python Prerequisites
[Montag](montag.py) requires Python 3 and the [EbookLib](https://github.com/aerkalov/ebooklib) and [python-magic](https://github.com/ahupp/python-magic) libraries. It also uses some utilities from the [Calibre](https://calibre-ebook.com/) project.
On a Debian-based Linux distribution, these requirements could be installed with:
```
$ sudo apt-get install libmagic1 imagemagick calibre-bin python3 python3-magic python3-ebooklib
```
On Windows, you'll need DLLs for `libmagic`. One option for installing these libraries is [`python-magic-bin`](https://pypi.org/project/python-magic-bin/):
```
python3 -m pip install python-magic-bin
```
The Python dependencies *should* be installed automatically if you are using `pip` to install montag.
### Docker
Alternately, a [Dockerfile](./docker/Dockerfile) is provided to allow you to run Montag in Docker. You can build the `oci.guero.org/montag:latest` Docker image with [`build_docker.sh`](./docker/build_docker.sh), then use [`montag-docker.sh`](./docker/montag-docker.sh) to process your e-book files.
## Usage
Montag is easy to use. Specify the input and output e-book filenames, and, optionally, the file containing the words to be censored (one per line) and the text encoding.
```
$ ./montag.py
usage: montag.py [options]
e-book profanity scrubber
required arguments:
-i , --input
Input file
-o , --output
Output file
-w , --word-list
Profanity list text file (default: swears.txt)
-e , --encoding
Text encoding (default: utf-8)
```
So, using Andy Weir's "The Martian" as an example:
```
$ ./montag.py -i "The Martian - Andy Weir.mobi" -o "The Martian - Andy Weir (scrubbed).mobi"
Processing "The Martian - Andy Weir.mobi" of type "Mobipocket E-book "The Martian", 775003 bytes uncompressed, version 6, codepage 65001"
Extracting metadata...
Converting to EPUB...
Processing book contents...
Generating output...
Converting...
Restoring metadata...
```
Upon opening the book, you will find the text reads something like this:
> CHAPTER 1
>
> LOG ENTRY: SOL 6
>
> I’m pretty much ******.
>
> That’s my considered opinion.
>
> ******.
>
> Six days into what should be the greatest two months of my life, and it’s turned into a nightmare.
>
> ...
Alternately, if you are using the Docker method described above, use [`montag-docker.sh`](./docker/montag-docker.sh) rather than [`montag.py`](./src/montag_cleaner/montag.py) directly.
## Known Limitations
Montag is not smart enough to do any in-depth language analysis or deep filtering. For a while I was trying to use the [rominf/profanity-filter](https://github.com/rominf/profanity-filter) library for the word detection and filtering, but I ran into issues and ended up just going with a simpler method that works but presents a few limitations:
* Only whole words are matched and censored. In other words, if the word `frick` is in your list of profanity, `Frick you!` will be censored, but `Absofrickenlutely` will not. As such if you wish to catch all of the variations of the word `frick`, you'd have to list them individually in your `swears.txt` word list.
* Having phrases (eg., multiple space-separated words) in your `swears.txt` word list won't do you any good.
* Montag can't tell the difference between different meanings of the same word. For example, if the word `ass` is in your list, both "And he said unto his sons, Saddle me the ass. So they saddled him the ass: and he rode thereon" (from the KJV of *The Bible*) and "Then the high king carefully turned the golden screw. Once: Nothing. Twice: Nothing. Then he turned it the third time, and the boy’s ass fell off" (from Patrick Rothfuss' *The Wise Man's Fear*) will be censored.
## Contributing
If you'd like to help improve Montag, pull requests will be welcomed!
## Authors
* **Seth Grover** - *Initial work* - [mmguero](https://github.com/mmguero)
## License
This project is licensed under the BSD 3-Clause License - see the [LICENSE](LICENSE) file for details.
## Acknowledgments
Thanks to:
* [Calibre](https://calibre-ebook.com/about) developer Kovid Goyal and contributors
* the contributors to [EbookLib](https://github.com/aerkalov/ebooklib/blob/master/AUTHORS.txt)
* [python-magic](https://github.com/ahupp/python-magic) developer Adam Hupp and contributors