{"id":18150918,"url":"https://github.com/005-bot/monitor","last_synced_at":"2025-04-06T23:28:03.313Z","repository":{"id":258959233,"uuid":"875052441","full_name":"005-bot/monitor","owner":"005-bot","description":"Сервис выполняет периодическое сканирование страницы с актуальными отключениями, выявляет изменения и отправляет их в Redis PubSub.","archived":false,"fork":false,"pushed_at":"2024-10-24T07:18:30.000Z","size":40,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2024-10-25T03:33:00.444Z","etag":null,"topics":["beautifulsoup","httpx","monitoring","mvp","pipenv","pubsub","python","redis","web-scraping"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/005-bot.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-19T01:38:54.000Z","updated_at":"2024-10-24T07:18:33.000Z","dependencies_parsed_at":"2024-10-26T10:09:22.424Z","dependency_job_id":"35ce9d55-e862-43de-83eb-c9a60ae06dbe","html_url":"https://github.com/005-bot/monitor","commit_stats":null,"previous_names":["005-bot/monitor"],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/005-bot%2Fmonitor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/005-bot%2Fmonitor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/005-bot%2Fmonitor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/005-bot%2Fmonitor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/005-bot","download_url":"https://codeload.github.com/005-bot/monitor/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247568153,"owners_count":20959746,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["beautifulsoup","httpx","monitoring","mvp","pipenv","pubsub","python","redis","web-scraping"],"created_at":"2024-11-02T01:05:51.882Z","updated_at":"2025-04-06T23:28:03.295Z","avatar_url":"https://github.com/005-bot.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 005 Бот - Монитор\n\nСервис выполняет периодическое сканирование страницы с актуальными отключениями, выявляет изменения и отправляет их в Redis PubSub.\n\nПроект находится на стадии MVP и может содержать обратно несовместимые изменения.\n\n## Используемые технологии, библиотеки\n\n- [Python](https://www.python.org/)\n- [Docker](https://www.docker.com/)\n- [Pipenv](https://github.com/pypa/pipenv)\n- [Redis](https://redis.io/)\n- [BeautifulSoup](https://www.crummy.com/software/BeautifulSoup/)\n- [httpx](https://github.com/encode/httpx/)\n\n## Настройки\n\nДля настройки используются переменные окружения:\n\n| Название            | Описание                              | По умолчанию                        |\n| ------------------- | ------------------------------------- | ----------------------------------- |\n| `REDIS__URL`        | URL Redis                             | `redis://localhost:6379`            |\n| `SCRAPER__URL`      | URL страницы с отключениями           | `http://93.92.65.26/aspx/Gorod.htm` |\n| `SCRAPER__INTERVAL` | Период проверки обновлений в секундах | `60`                                |\n| `STORAGE__TTL_DAYS` | Время хранения хэшей записей в днях   | `5`                                 |\n| `STORAGE__PREFIX`   | Префикс хранилища для ключей в Redis  | `bot-005`                           |\n| `PUBLISHER__PREFIX` | Префикс очереди PubSub в Redis        | `bot-005`                           |\n\n\n## Логика работы\n\n1. Сервис с периодичностью в `SCRAPER__INTERVAL` секунд запрашивает страницу по адресу `SCRAPER__URL`.\n2. Полученная страница разбирается на отдельные записи.\n3. Для каждой записи считается хэш и сравнивается с сохраненными в хранилище значениями.\n4. Записи, отсутствующие в хранилище считаются новыми и отправляются в очередь сообщений.\n5. Новые значения хэшей записываются в хранилище.\n\n## Лицензия\n\nПроект распространяется под лицензией Apache 2.0.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F005-bot%2Fmonitor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F005-bot%2Fmonitor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F005-bot%2Fmonitor/lists"}