{"id":24091392,"url":"https://github.com/bateman/tpscanner-cli","last_synced_at":"2025-10-28T14:08:08.374Z","repository":{"id":224827190,"uuid":"738037247","full_name":"bateman/tpscanner-cli","owner":"bateman","description":"A utility script to find the best cumulative price for items listed under Trovaprezzi","archived":false,"fork":false,"pushed_at":"2024-12-24T11:26:13.000Z","size":55913,"stargazers_count":1,"open_issues_count":3,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-10T07:13:04.082Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://bateman.github.io/tpscanner-cli/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bateman.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-02T09:10:29.000Z","updated_at":"2024-12-24T11:26:10.000Z","dependencies_parsed_at":"2024-04-12T13:56:10.393Z","dependency_job_id":"03f48a45-89a9-401a-ae69-179b72bf4fb7","html_url":"https://github.com/bateman/tpscanner-cli","commit_stats":null,"previous_names":["bateman/tpscanner-cli"],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bateman%2Ftpscanner-cli","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bateman%2Ftpscanner-cli/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bateman%2Ftpscanner-cli/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bateman%2Ftpscanner-cli/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bateman","download_url":"https://codeload.github.com/bateman/tpscanner-cli/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240993389,"owners_count":19890418,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-10T07:13:14.786Z","updated_at":"2025-10-28T14:08:03.340Z","avatar_url":"https://github.com/bateman.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n\n```\n _________\n/_  __/ _ \\  __ ______ ____  ___  ___ ____\n / / / ___/\\ \\/ __/ _ `/ _ \\/ _ \\/ -_) __/\n/_/ /_/  /___/\\__/\\_,_/_//_/_//_/\\__/_/\n```\n#\n\n![GitHub Actions Workflow Status](https://img.shields.io/github/actions/workflow/status/bateman/tpscanner-cli/release.yml?style=flat-square)\n![GitHub Release](https://img.shields.io/github/v/release/bateman/tpscanner-cli?style=flat-square)\n![GitHub language count](https://img.shields.io/github/languages/count/bateman/tpscanner-cli?style=flat-square)\n![GitHub top language](https://img.shields.io/github/languages/top/bateman/tpscanner-cli?style=flat-square)\n![GitHub code size in bytes](https://img.shields.io/github/languages/code-size/bateman/tpscanner-cli?style=flat-square)\n![Codecov](https://img.shields.io/codecov/c/github/bateman/tpscanner-cli?style=flat-square)\n![GitHub License](https://img.shields.io/github/license/bateman/tpscanner-cli?style=flat-square)\n\n\n***TPscanner*** is a Python script that extracts prices of items from [Trovaprezzi.it](https://www.trovaprezzi.it/), sorts them, displays and saves the results in a spreadsheet. It also finds the best cumulative and individual deals.\n\nIf your don't want to use the command line, check the [TPscanner browser extension](https://github.com/bateman/tpscanner-cli). It works on Chromium-based browsers (e.g., Chrome, Edge), Firefox, and Safari.\n\n![Intro Image](img/intro.gif)\n\n## Setup\n\nBefore you can run TPScanner, you need to set up your environment. This project uses [Poetry](https://python-poetry.org/) for dependency management. If you haven't installed Poetry yet, you can do so by following the instructions on their [official website](https://python-poetry.org/docs/#installation).\n\nOnce you have Poetry installed, follow these steps to set up the project:\n\n1. Clone the repository:\n\n        git clone https://github.com/yourusername/tpscanner-cli.git\n        cd tpscanner-cli\n\n2. Createa and activate a virtual environment (`pyenv` is recommended).\n\n3. Install the project dependencies:\n\n        make install\n\n\n### External dependencies\n\nThe script relies on [Selenium](https://www.selenium.dev/) web driver. Make sure that the Chrome/Chromium web browser is installed before running the script.\n\n### Note\n\nIf you don't have `poetry` installed (or don't want to install it), you can use `pip` as follows:\n\n1. First, create a virual environment: `python -m venv .tps`.\n2. Activate it: `source .tps/bin/activate`.\n3. Install requirements: `pip install -r requirements.txt`.\n4. Optional, for development purposes only, run also: `pip install -r requirements-dev.txt`.\n\n\n## Usage\n\nTo run the script, use the following command:\n\n```bash\npython -m tpscanner -u url1 url2 ... | -f path/to/input/file.txt [-q n1 n2 ...] [--includena] [-w n] [--headless] [--console] [--excel]\n```\n```console\noptions:\n  -h, --help              Show this help message and exit\n  -u URL [URL ...], --url URL [URL ...]\n                          List of URLs to scan\n  -f FILE, --file FILE    File containing URLs to scan\n  -q QUANTITY [QUANTITY ...], --quantity  QUANTITY [QUANTITY ...]\n                          List of quantities to buy for each URL (in order)\n  -i , --includena        Whether to include items marked as not available\n  -w WAIT, --wait WAIT    Wait time between URLs requests (default 5 sec.)\n  --headless              Run in headless mode\n  -c, --console           Whether to print results to the console\n  -x, --excel             Whether to save results to Excel\n  -l=LEVEL, --level=LEVEL Set the desired logging level\n                          (none, debug, info, warning, error, critical)\n```\n\nAlternatively, you can run the script as:\n\n```bash\npoetry run tpscanner ...\n```\n\nor\n\n```bash\nmake run ARGS=\"...\"\n```\n\n\u003e [!WARNING]\n\u003e The script can run with the browser in `headless` mode. In my tests, however, I've noticed that it often causes the server to display captchas, thus making the script scraping process fail.\n\n\n## Output\n\nWhen the `--console` option is enabled, the script outputs to the console\nthe results in the form of tables.\n\nWhen the `--excel` option is enabled, the script creates a spreadsheet named `results_\u003ccurrent_datetime\u003e.xlsx` with the sorted list of items and the best cumulative deals.\n\n## Configuration\n\nYou can configure the script by editing the file `config/config.json`. At the moment, you can configure:\n\n- `sleep_rate_limit = 2`: Too aggressive scraping will cause the server to show captchas. By default, the script will wait 2 secs. in between each item's offer scraping.\n- `chrome_version: 120`: The Chrome version to use with the undetected_chromdriver module.\n- `user_agents = []`: A list of browser User-Agent strings to cycle through in headless mode.\n- `output_dir = results`: The output directory where to store the Excel output file. It is set to the `results/` subfolder in the current working directory by default.\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](https://raw.githubusercontent.com/bateman/tpscanner-cli/main/LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbateman%2Ftpscanner-cli","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbateman%2Ftpscanner-cli","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbateman%2Ftpscanner-cli/lists"}