https://github.com/kokseen1/wall-e
A self-hosted Carousell scraper.
https://github.com/kokseen1/wall-e
carousell docker docker-compose heroku postgres postgresql python python-telegram-bot telegram telegram-bot
Last synced: 7 months ago
JSON representation
A self-hosted Carousell scraper.
- Host: GitHub
- URL: https://github.com/kokseen1/wall-e
- Owner: kokseen1
- Created: 2022-03-28T07:03:07.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2022-09-08T16:27:41.000Z (about 3 years ago)
- Last Synced: 2025-01-13T06:42:06.705Z (9 months ago)
- Topics: carousell, docker, docker-compose, heroku, postgres, postgresql, python, python-telegram-bot, telegram, telegram-bot
- Language: Python
- Homepage:
- Size: 31.3 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
#  WALL-E
WALL-E is a self-hosted Carousell scraper that provides an intuitive Telegram interface via the Telegram Bot API.
WALL-E is easily deployed via Docker Compose either on your local machine or on Heroku for free. Configurations such as the polling frequency can be tweaked by modifying the Python source code.
## Prerequisites
- [Docker](https://docs.docker.com/get-docker/)
- [Docker Compose](https://docs.docker.com/compose/install/)
- An available [Telegram Bot](https://t.me/botfather/)
### Deploying on Heroku
- [Heroku CLI](https://devcenter.heroku.com/articles/heroku-cli)
- [psql](https://www.postgresql.org/download/)
## Installation
Clone this repository
```shell
git clone https://github.com/kokseen1/wall-e.git
cd wall-e
```
## Local Deployment
Create a `.env` file
```shell
touch .env
```
Set environment variables by adding the following lines in `.env`
```
BOT_TOKEN=
POSTGRES_NAME=postgres
POSTGRES_USER=postgres
POSTGRES_PASSWORD=postgres
NO_SSL_VERIFY=True
```
Build and start the containers
```shell
sudo docker-compose up
```
## Heroku Deployment
Login to Heroku
```shell
heroku login
```
Create a Heroku app and add the Git remote
```shell
heroku create -a
heroku git:remote -a
```
Set the stack of your app to `container`
```shell
heroku stack:set container
```
Add the Heroku `postgresql` addon
```shell
heroku addons:create heroku-postgresql:hobby-dev
```
Set environment variables
```shell
heroku config:set HEROKU_URL=https://.herokuapp.com/
heroku config:set BOT_TOKEN=
heroku config:set POSTGRES_NAME=postgres
heroku config:set POSTGRES_USER=postgres
heroku config:set POSTGRES_PASSWORD=postgres
```
Connect to the database via `psql`
```shell
heroku pg:psql
```
Create tables
```sql
CREATE TABLE sent (
item_id VARCHAR ( 255 ) NOT NULL,
chat_id VARCHAR ( 255 ) NOT NULL
);
CREATE TABLE queries (
query_text VARCHAR ( 255 ) NOT NULL,
chat_id VARCHAR ( 255 ) NOT NULL
);
```
Exit from `psql`
```shell
exit
```
Push to Heroku
```shell
git push heroku main
```
Keep the application awake via a service like [Kaffeine](https://kaffeine.herokuapp.com/)
## Bot Usage
Add query
```
/add
```
Remove query
```
/rm
```
List queries
```
/ls
```
Fetch queries
```
/force
```