https://github.com/itallix/mipt-hack-23
Telegram bot powered by Generative AI for MIPT Hackathon 23.
https://github.com/itallix/mipt-hack-23
Last synced: about 1 month ago
JSON representation
Telegram bot powered by Generative AI for MIPT Hackathon 23.
- Host: GitHub
- URL: https://github.com/itallix/mipt-hack-23
- Owner: itallix
- License: mit
- Created: 2023-12-14T22:58:30.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2023-12-27T19:56:10.000Z (over 1 year ago)
- Last Synced: 2025-02-12T10:27:15.201Z (3 months ago)
- Language: Jupyter Notebook
- Size: 4.9 MB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# MyCityGuideBot
tg: [@MyCityGuideBot](https://t.me/MyCityGuideBot)
A perfect assistant for urban adventurers and culture enthusiasts
**Features:**
- Responds with audio to voice message
- Generates description for uploaded picture
- `/story` - Discover an intriguing fact about a random city
- Supports 2 languages: English & Russian
Demo slide deck is available via [link](https://drive.google.com/file/d/1En0Qffk7cH2iQECuwknx1lKtRW8BZT4w/view?usp=sharing)
## Architecture
## Sequence diagrams
### Voice flow
### Image flow
## Prerequisites
1. [poetry](https://python-poetry.org/docs/)
```bash
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -
```
1. [ngrok](https://ngrok.com/)
```bash
brew install ngrok
```
1. [pre-commit](https://pre-commit.com/)
```bash
brew install pre-commit
pre-commit install
```
1. [gcloud](https://cloud.google.com/sdk/docs/install)
```bash
brew install --cask google-cloud-sdk
```
## Running jupyterlab
```bash
poetry install
poetry run jupyter lab
```
## Running bot locally
1. Make sure you have access to the Google project `mipt-hack-01` (contact tg: @fancyeagle if not)
```bash
gcloud auth application-default login
```
1. Register your bot for testing with [BotFather](https://t.me/BotFather) and save a telegram token as env variable:
```bash
export TELEGRAM_TOKEN=some_value
```
1. If you didn't create a virtual environment with poetry before, create it:
```bash
poetry install
```
1. Execute the following commands in terminal to start bot service locally:
```bash
export FLASK_APP=bot_service.app:app
export FLASK_RUN_PORT=5005
export FLASK_ENV=development
export PROJECT_ID=mipt-hack-01
poetry run flask run --reload
```
1. Create a tunnel for the local HTTP server from the previous step using `ngrok`, making it accessible from the
public internet:
```bash
ngrok http 127.0.0.1:5005
```
1. Using `sh/webhook.sh` script, register the webhook with Telegram API, so that the bot will trigger local service
on each request:
```bash
sh sh/webhook.sh set
```
- `PUBLIC_URL` - the url generated by ngrok on step 4, e.g.: `https://fce9-2a02-a447-94fc-1-50f5-8a25-6ab-eecc.ngrok.io`
- `TELEGRAM_TOKEN` - the token generated by BotFather on step 1
1. After you finished with testing, you can disconnect your bot using the following command:
```bash
sh sh/webook.sh delete
```
## Running BDD tests
To ensure the model accurately recognizes artwork, a set of Behavior-Driven Development
(BDD) tests has been created. These tests verify that the model's responses correctly
match the paintings and landmarks present in the photo.
Before running tests, download [data.zip](https://drive.google.com/file/d/15pBo5il84sa6OylYLhCTTbzAjabUDqNs/view?usp=drive_link)
and extract it to `tests/bdd` folder.
Running all tests:
```bash
poetry run pytest tests/bdd
```
Running tests for category:
```bash
poetry run pytest tests/bdd -k "paintings"
poetry run pytest tests/bdd -k "landmarks"
```
It appears that the model incorrectly identified the painting "The Stroll at Giverny"
by Claude Monet as "Woman and Children in a Field" by Berthe Morisot.
This misidentification can be due to several factors related to the model's training
and the inherent challenges in artwork recognition.