{"id":15009816,"url":"https://github.com/rcastellotti/yawgbot","last_synced_at":"2025-04-09T17:51:55.097Z","repository":{"id":138598029,"uuid":"530232472","full_name":"rcastellotti/yawgbot","owner":"rcastellotti","description":"a plugin-based tool to find accommodation","archived":false,"fork":false,"pushed_at":"2023-01-14T16:49:10.000Z","size":95,"stargazers_count":2,"open_issues_count":4,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-23T19:51:32.071Z","etag":null,"topics":["bot","plugin-system","python3"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rcastellotti.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-08-29T13:29:14.000Z","updated_at":"2024-05-20T08:14:30.000Z","dependencies_parsed_at":null,"dependency_job_id":"eb3ee81a-4e52-44cc-b101-fe481f7f7cdc","html_url":"https://github.com/rcastellotti/yawgbot","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rcastellotti%2Fyawgbot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rcastellotti%2Fyawgbot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rcastellotti%2Fyawgbot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rcastellotti%2Fyawgbot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rcastellotti","download_url":"https://codeload.github.com/rcastellotti/yawgbot/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248083182,"owners_count":21045055,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bot","plugin-system","python3"],"created_at":"2024-09-24T19:28:46.976Z","updated_at":"2025-04-09T17:51:55.047Z","avatar_url":"https://github.com/rcastellotti.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# yawgbot\n\n[![PyPI version](https://badge.fury.io/py/yawgbot.svg)](https://badge.fury.io/py/yawgbot)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n\nYet Another WG helper bot. WG stands for *Wohngemeinschaft*, a German word that refers to a living arrangement in which\nseveral tenants share an apartment. It is aimed to automate the extremely complex and time-consuming task of finding a\nroom or an apartment to rent. Initially it was only working for [wg-gesucht.de](https://wg-gesucht.de), now it's a\nplugin-based system ready to be extended for every platform to find accommodation.\n\nFrom a technical standpoint `yawgbot` is a simple tool that runs different plugins. A plugin either scrapes websites using [requests](https://requests.readthedocs.io/en/latest/) and [beautifulsoup4](https://beautiful-soup-4.readthedocs.io/en/latest/) or uses APIs (both public and undocumented) both to search for listings and to contact landlords. It uses a SQLite database to save contacted ads and to perform fewer requests to platforms. It also comes with a simple web UI to track your progress and gather your findings across different platforms.  \nThe database is stored:\n\n+ on GNU/Linux at `~/.local/share/yawgbot/yawgbot.sqlite`\n+ on Windows at `%USERPROFILE%\\AppData\\Local\\rcastellotti\\yawgbot\\yawgbot.sqlite`\n+ on macOS at `~/Library/Application Support/yawgbot/yawgbot.sqlite`\n\n## running manually\n\nTo run the bot manually:\n\n+ install the package with `pip3 install yawgbot` (a [virtual environment](https://docs.python.org/3/tutorial/venv.html)\n  is suggested)\n+ configure `.config.myl`, stored\n    + on GNU/Linux at `~/.config/yawgbot/config.yml`\n    + on Windows at `%USERPROFILE%\\AppData\\Local\\rcastellotti\\yawgbot\\config.yml`\n    + on macOS at `~/Library/Preferences/yawgbot/config.yml`\n+ create a new file named `bot.py` with the following content and run it:\n\n```python\nfrom yawgbot import Bot\nimport logging\n\nlogging.basicConfig(level=logging.INFO)\nbot = Bot(platforms=[\"wg-gesucht\"])\nbot.run()\n```\n\n## running periodically with celery\n\nRunning in background at time intervals is achieved using [Celery](https://docs.celeryq.dev/en/stable/) to schedule tasks. It is configured to use [SQLite](https://sqlite.org) as both [backend and broker](https://docs.celeryq.dev/en/stable/getting-started/backends-and-brokers/index.html), to know\nmore read the docs. The file `run_yawgbot.py`, for example, runs the bot each 5 minutes, to use it:\n\n- run `celery -A run_yawgbot.celery worker --loglevel=INFO` to run the worker\n- run `celery -A run_yawgbot.celery beat` to schedule the bot\n\n## web UI\n\nTo start the web UI simply run `yawgbot-web`\n\n## plugins\n\n### creating a plugin\n\nExtending `yawgbot` is a pretty straightforward process. A plugin is a simple module extending the `PluginBase` class, an example is [`wg-gesucht.py`](https://github.com/rcastellotti/yawgbot/blob/master/src/yawgbot/plugins/wg-gesucht.py),\nsupported plugins can be used just by instantiating `Bot` with the `platforms` argument. Custom plugins can be registered using the `Bot.register_plugin()` method.  \nTo start creating a template you can use this boilerplate code:\n\n```python3\nfrom yawgbot.pluginBase import PluginBase\n\n\nclass YawgbotPlugin(PluginBase):\n    def __init__(self):\n        pass\n\n    def get_ads(self, url):\n        pass\n\n    def parse_ad(self, ad):\n        pass\n\n    def contact_ad(self, slug):\n        pass\n\n    def run(self):\n        pass\n```\n\nIf you develop a plugin, consider creating a PR, I will be more than happy to work with you to make it an official plugin and ship it with `yawgbot`.\n\nEach plugin should load configuration values using the static `Bot.config()` method.   \nTo better understand plugins you can read the code in [`src/yawgbot/plugins`](https://github.com/rcastellotti/yawgbot/tree/master/src/yawgbot/plugins).\n\n### wg-gesucht\n\n#### config \n```yml\nwg-gesucht:\n  username: \"\"\n  password: \"\"\n  message_template: \"\"\n  base_url: \"\"\n```\n\nWhen configuring for use go on [https://wg-gesucht.de](https://wg-gesucht.de) and copy the url you are using to look for accommodation, then replace the last number with `{}` in order to be able to search across multiple pages, for example, if looking for apartments in Munich: base url is `https://www.wg-gesucht.de/1-zimmer-wohnungen-in-Munchen.90.1.1.1.html` , replace the last `1` as follows: `https://www.wg-gesucht.de/1-zimmer-wohnungen-in-Munchen.90.1.1.{}.html`","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frcastellotti%2Fyawgbot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frcastellotti%2Fyawgbot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frcastellotti%2Fyawgbot/lists"}