https://github.com/sralloza/parsers

https://github.com/sralloza/parsers

Last synced: about 1 month ago
JSON representation

• Host: GitHub
• URL: https://github.com/sralloza/parsers
• Owner: sralloza
• Created: 2021-03-29T13:08:04.000Z (over 3 years ago)
• Default Branch: master
• Last Pushed: 2022-03-14T12:47:49.000Z (almost 3 years ago)
• Last Synced: 2023-03-09T04:32:34.735Z (almost 2 years ago)
• Language: Python
• Size: 67.4 KB
• Stars: 0
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Parsers

**_WARNING: this project is deprecated. Please use [sralloza/entertainment-source-manager](https://github.com/sralloza/entertainment-source-manager) instead._**

Finds new anime and manga chapters and sends a notification via telegram.

## Production

Required settings:

- `TELEGRAM_BOT_TOKEN`: telegram bot token to send telegram notifications.

- `TELEGRAM_CHAT_ID`: telegram chat id to send messages to.

- `S3_BUCKET_NAME`: AWS S3 bucket name to store the processed chapters.

- `AWS_ACCESS_KEY_ID`: AWS access key ID.

- `AWS_SECRET_ACCESS_KEY` AWS secret access key.

Optional settings:

- `MANGA_CONFIG_PATH`: full path to the `manga.json` file. Must be the container's path, not the host's path. **Note: the container's path can't be set to `/code` because the code is there.**

- `PARSE_ONE_PIECE_ANIME`: boolean, false by default. It enables the one piece anime parsing. **WARNING: one piece anime parsing is currently broken, do not enable it**.

- `TODOIST_TOKEN`: todoist token to enable todoist integration.

- `TODOIST_PROJECT_ID`: todoist project id.

- `TODOIST_DUE_STR`: string, `today` by default. Set the due string for the todoist tasks.

## Development

**Note: these are the commands used to build the docker images. This should not be used. Docker images are built automatically using the [publish pipeline](.github/workflows/publish.yml).**

```shell

docker buildx build -t sralloza/parsers:$VERSION --platform=linux/arm/v7,linux/amd64,linux/arm64 --push .

```

To run:

```shell

docker run --env-file ENV_FILE --rm sralloza/parsers:$VERSION parse

```

To parse mangas, create a file `mangas.json`:

```json

{

  "manga-name": "first-chapter-uuid"

}

```

Then launch the `docker run` command appending the `mangas.json` file as a volume. The `MANGA_CONFIG_PATH` environment variable must be set accordingly

**Note: the manga name in the json file must be written as lowercase without spaces. The name will be title-cased later.**

### AWS-cli utils

- Create bucket: `aws s3api create-bucket --bucket $BUCKET`

- Delete bucket: `aws s3api delete-bucket --bucket $BUCKET`

- List buckets: `aws s3api list-buckets`

- Show data in bucket: `aws s3api list-objects --bucket $BUCKET --query 'Contents[].{Key: Key, Size: Size}'`

- Remove file in bucket: `aws s3 rm s3://$BUCKET/$FILE`

- Download file from bucket: `aws s3 cp s3://$BUCKET/$FILE ./`

- Upload file to bucket: `aws s3 cp $FILE s3://$BUCKET/$FILE`