Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/lzzy12/python-aria-mirror-bot

A telegram bot for all your mirror needs | OG Repo
https://github.com/lzzy12/python-aria-mirror-bot

google-drive mirror mirror-bot python-mirror-bot team-drive telegram-bot telegram-mirror-bot

Last synced: 5 days ago
JSON representation

A telegram bot for all your mirror needs | OG Repo

Awesome Lists containing this project

README 

        
# What is this repo about?

This is a telegram bot writen in python for mirroring files on the internet to our beloved Google Drive.



# Inspiration 

This project is heavily inspired from @out386 's telegram bot which is written in JS.



# Features supported:

- Mirroring direct download links to google drive

- Download progress

- Upload progress

- Download/upload speeds and ETAs

- Docker support

- Uploading To Team Drives.

- Index Link support

- Service account support

- Mirror all youtube-dl supported links

- Mirror telegram files

- Stable Mega.nz support

- Unzip downloads

- Persistent authorised chat storage



# How to deploy?

Deploying is pretty much straight forward and is divided into several steps as follows:

## Installing requirements



- Clone this repo:

```

git clone https://github.com/lzzy12/python-aria-mirror-bot mirror-bot/

cd mirror-bot

```



- Install requirements

For Debian based distros

```

sudo apt install python3

```

Install Docker by following the [official docker docs](https://docs.docker.com/engine/install/debian/)



- For Arch and it's derivatives:

```

sudo pacman -S docker python

```



- Install dependencies for running setup scripts:

```shell script

pip3 install -r requirements-cli.txt

```



## Setting up config file

NOTE: Starting from 953113269e4424ecb956d0b9b093b846301efffb we are moving away from storing authroized chat ids in txt files. We will be using a light weight redis db. So, going forword, a redis db is a must.

```

cp config_sample.env config.env

```

- Remove the first line saying:

```

_____REMOVE_THIS_LINE_____=True

```

Fill up rest of the fields. Meaning of each fields are discussed below:

- **BOT_TOKEN** : The telegram bot token that you get from @BotFather

- **GDRIVE_FOLDER_ID** : This is the folder ID of the Google Drive Folder to which you want to upload all the mirrors.

- **DOWNLOAD_DIR** : The path to the local folder where the downloads should be downloaded to

- **DOWNLOAD_STATUS_UPDATE_INTERVAL** : A short interval of time in seconds after which the Mirror progress message is updated. (I recommend to keep it 5 seconds at least)  

- **OWNER_ID** : The Telegram user ID (not username) of the owner of the bot

- **AUTO_DELETE_MESSAGE_DURATION** : Interval of time (in seconds), after which the bot deletes it's message (and command message) which is expected to be viewed instantly. Note: Set to -1 to never automatically delete messages

- **IS_TEAM_DRIVE** : (Optional field) Set to "True" if GDRIVE_FOLDER_ID is from a Team Drive else False or Leave it empty.

- **USE_SERVICE_ACCOUNTS**: (Optional field) (Leave empty if unsure) Whether to use service accounts or not. For this to work see  "Using service accounts" section below.

- **INDEX_URL** : (Optional field) Refer to https://github.com/maple3142/GDIndex/ The URL should not have any trailing '/'

- **TELEGRAM_API** : This is to authenticate to your telegram account for downloading Telegram files. You can get this from https://my.telegram.org DO NOT put this in quotes.

- **TELEGRAM_HASH** : This is to authenticate to your telegram account for downloading Telegram files. You can get this from https://my.telegram.org

- **USER_SESSION_STRING** : Session string generated by running:

- **MEGA_KEY**: Mega.nz api key to mirror mega.nz links. Get it from [Mega SDK Page](https://mega.nz/sdk)

- **MEGA_USERNAME**: Your mega email id (You can leave it empty, it will start megasdkrest server in anonymous mode)

- **MEGA_PASSWORD**: Your password for your mega.nz account. (**NOTE**: You must deactivate 2FA to use the account with the bot otherwise bot will not be able to sign in)

- **REDIS_HOST**: Redis DB Host URL. Redis DB is used to store the authorised chats informations (you can get it at https://redis.com -> Login -> Select Free subscription -> Create Database -> Enter a name -> Select a location(if you want) -> keep everything else default. You will get a public endpoint(refered to as REDIS_HOST in this repo), port (most probably 6397) and a password)

- **REDIS_PORT**: Redis DB Port number.

- **REDIS_PASSWORD**: Password of your redis DB

```

python3 generate_string_session.py

```

Note: You can limit maximum concurrent downloads by changing the value of MAX_CONCURRENT_DOWNLOADS in aria.sh. By default, it's set to 2

 

## Getting Google OAuth API credential file



- Visit the [Google Cloud Console](https://console.developers.google.com/apis/credentials)

- Go to the OAuth Consent tab, fill it, and save.

- Go to the Credentials tab and click Create Credentials -> OAuth Client ID

- Choose Other and Create.

- Use the download button to download your credentials.

- Move that file to the root of mirror-bot, and rename it to credentials.json

- Visit [Google API page](https://console.developers.google.com/apis/library)

- Search for Drive and enable it if it is disabled

- Finally, run the script to generate token file (token.pickle) for Google Drive:

```

pip install google-api-python-client google-auth-httplib2 google-auth-oauthlib

python3 generate_drive_token.py

```

## Deploying



- Start docker daemon (skip if already running):

```

sudo dockerd

```

- Build Docker image:

```

sudo docker build . -t mirror-bot

```

- Run the image:

```

sudo docker run mirror-bot

```



# Using service accounts for uploading to avoid user rate limit

For Service Account to work, you must set USE_SERVICE_ACCOUNTS="True" in config file or environment variables

Many thanks to [AutoRClone](https://github.com/xyou365/AutoRclone) for the scripts

**NOTE:** Using service accounts is only recommended while uploading to a team drive.

## Generating service accounts

Step 1. Generate service accounts [What is service account](https://cloud.google.com/iam/docs/service-accounts)

---------------------------------

Let us create only the service accounts that we need. 

**Warning:** abuse of this feature is not the aim of this project and we do **NOT** recommend that you make a lot of projects, just one project and 100 sa allow you plenty of use, its also possible that over abuse might get your projects banned by google. 



```

Note: 1 service account can copy around 750gb a day, 1 project can make 100 service accounts so that's 75tb a day, for most users this should easily suffice. 

```



`python3 gen_sa_accounts.py --quick-setup 1 --new-only`



A folder named accounts will be created which will contain keys for the service accounts



NOTE: If you have created SAs in past from this script, you can also just re download the keys by running:

```

python3 gen_sa_accounts.py --download-keys project_id

```



### Add all the service accounts to the Team Drive

- Run:

```

python3 add_to_team_drive.py -d SharedTeamDriveSrcID

```



# Youtube-dl authentication using .netrc file

For using your premium accounts in youtube-dl, edit the netrc file (in the root directory of this repository) according to following format:

```

machine host login username password my_youtube_password

```

where host is the name of extractor (eg. youtube, twitch). Multiple accounts of different hosts can be added each separated by a new line