{"id":19965804,"url":"https://github.com/allendema/rewe_dl","last_synced_at":"2025-08-21T20:07:45.570Z","repository":{"id":251258915,"uuid":"836433132","full_name":"allendema/rewe_dl","owner":"allendema","description":"Call store APIs. Parse responses and save the data to SQL, JSON or any custom format.","archived":false,"fork":false,"pushed_at":"2025-02-25T22:38:08.000Z","size":599,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-25T23:30:36.596Z","etag":null,"topics":["apprise","discounts","httpx","json","jsonl","market","price-comparison","price-tracker","products","python3","ruff","sqlite3","webhooks"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/allendema.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-07-31T20:52:14.000Z","updated_at":"2025-02-25T22:38:11.000Z","dependencies_parsed_at":"2025-02-04T23:18:13.938Z","dependency_job_id":"dfe1533d-e5c0-46d9-9532-3acdd97d485b","html_url":"https://github.com/allendema/rewe_dl","commit_stats":null,"previous_names":["allendema/rewe_dl"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allendema%2Frewe_dl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allendema%2Frewe_dl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allendema%2Frewe_dl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allendema%2Frewe_dl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/allendema","download_url":"https://codeload.github.com/allendema/rewe_dl/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241396781,"owners_count":19956408,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apprise","discounts","httpx","json","jsonl","market","price-comparison","price-tracker","products","python3","ruff","sqlite3","webhooks"],"created_at":"2024-11-13T02:31:14.744Z","updated_at":"2025-03-01T17:17:20.362Z","avatar_url":"https://github.com/allendema.png","language":"Python","readme":"## About\nPython library to call the APIs that the store itself calls.  \nTo get good parsed results from those API calls see [`parser.py`](https://github.com/allendema/rewe_dl/blob/main/rewe_dl/parser.py).  \n\n[![Python Version from PEP 621 TOML](https://shields.sp-codes.de/python/required-version-toml?tomlFilePath=https%3A%2F%2Fraw.githubusercontent.com%2Fallendema%2Frewe_dl%2Fmain%2Fpyproject.toml)](https://github.com/allendema/rewe_dl/main/pyproject.toml)\n[![Formatter](https://shields.sp-codes.de/badge/formatter-ruff-blue)](https://github.com/astral-sh/ruff)\n[![github commits](https://shields.sp-codes.de/github/last-commit/allendema/rewe_dl)](https://github.com/allendema/rewe_dl/commits/main)\n\n## Usage\n\u003cdetails\u003e\n    \u003csummary\u003ePrerequisites\u003c/summary\u003e\n\n    sudo apt-get install git\n    python -m ensurepip --upgrade\n\n\u003c/details\u003e\n\n```bash\ngit clone https://github.com/allendema/rewe_dl \u0026\u0026 cd rewe_dl\n```  \n```bash\npip install requirements.txt\n```  \n```bash\npip install -e rewe_dl\n```\n\nOptional [apprise](https://github.com/caronc/apprise/wiki/) can be used to send send notifications to more than 100 services/apps.\n\nSee [Config](https://github.com/allendema/rewe_dl/main/README.md#config) options!\n\n\n## Examples\nYou can:\n  - save product infos to JSON\n  - save product infos to JSONL\n  - save product infos to SQL\n  - save product images\n  - save product calories or other details\n  - monitor your _basket_ with your fav products\n  - compare prices and get notified when prices change\n  - save products to SQL for further analysis\n  - analyse the output data the way you like, for example: inflation analysis.\n  - _and_ whatever you want.\n\n\n\u003cdetails\u003e\n    \u003csummary\u003eexamples/discounted_to_json.py\u003c/summary\u003e\n    https://github.com/allendema/rewe_dl/blob/7948de8ebc401af4791cdd66fd928daac58d749e/rewe_dl/examples/discounted_to_json.py#L26-L37\n\u003c/details\u003e\n\n\u003cdetails\u003e\n    \u003csummary\u003eexamples/discounted_to_sql.py\u003c/summary\u003e\n    https://github.com/allendema/rewe_dl/blob/7948de8ebc401af4791cdd66fd928daac58d749e/rewe_dl/examples/discounted_to_sql.py#L23-L35\n\u003c/details\u003e\n\n\u003cdetails\u003e\n    \u003csummary\u003eexamples/my_basket.py\u003c/summary\u003e\n    https://github.com/allendema/rewe_dl/blob/7948de8ebc401af4791cdd66fd928daac58d749e/rewe_dl/examples/my_basket.py#L23-L37\n\u003c/details\u003e\n\n\u003cdetails\u003e\n    \u003csummary\u003eexamples/newest_products_to_sql.py\u003c/summary\u003e\n    https://github.com/allendema/rewe_dl/blob/7948de8ebc401af4791cdd66fd928daac58d749e/rewe_dl/examples/newest_products_to_sql.py#L19-L26\n\u003c/details\u003e\n\n\u003cdetails\u003e\n    \u003csummary\u003eexamples/notify_when_price.py\u003c/summary\u003e\n    https://github.com/allendema/rewe_dl/blob/7948de8ebc401af4791cdd66fd928daac58d749e/rewe_dl/examples/notify_when_price.py#L54-L66\n\u003c/details\u003e\n\n\u003cdetails\u003e\n    \u003csummary\u003eexamples/raw_responses_to_json.py\u003c/summary\u003e\n    https://github.com/allendema/rewe_dl/blob/7948de8ebc401af4791cdd66fd928daac58d749e/rewe_dl/examples/raw_responses_to_json.py#L26-L43\n\u003c/details\u003e\n\n\nSee `rewe_dl/examples` folder for some more starting examples to embedd it into your own project.\n\n\n## Config\n### Getting the cookies\nDepending on store location, some stores will have different prices.\n\nTo not use the default store, go to [https://shop.rewe.de](https://shop.rewe.de)  \nand select your pickup store.\n\nRight click -\u003e Inspect source -\u003e Web Storage -\u003e Cookies  \nand copy the value from `marketsCookie` into the `config.json` file.\n\n\n### Notifications/Webhooks\nIf you don't want to install apprise, you can use [matrix.org](https://matrix.org) or [telegram.org](https://telegram.org).  \n\n- Matrix:\n    Fill the `matrix` dict in `rewe_dl/config.json` file.\n- Telegram:\n    Fill the `telegram` dict in `rewe_dl/config.json` file.\n- Apprise (Optional): The config file in `~/.config/apprise` (without any extension) will be used.  \n    Configure your wanted notifications/webhooks options based on the [wiki](https://github.com/caronc/apprise/wiki/).\n\n\n## Changes\nAPI endpoints or data structure might change!\nIf you have any issues report them or open a pull request.\n\nThis project will be a part of another project which uses multiple stores using an REST API with Fastapi.\n\n\n\u003cdetails\u003e\n\n\u003csummary\u003eDevelopment\u003c/summary\u003e\n\n### Development\nIf you want to modify something:  \n  - `pip install requirements-dev.txt`\n  - The `pyproject.toml` file is used for linting/formatting with `ruff`.\n  - Make your changes.\n  - If you add tests, run them with `python3 ./scripts/run_tests.py`.\n  - Run `ruff check --fix .`\n  - Run `ruff format .`\n  - Create a [Pull Request](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request).\n\n\u003c/details\u003e\n\n\n## Credits\nInspired from [gallery-dl](https://github.com/mikf/gallery-dl), the code structure is similiar and some is code straight reused.  \nWhere applicable - docstrings point to original code creators.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fallendema%2Frewe_dl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fallendema%2Frewe_dl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fallendema%2Frewe_dl/lists"}