{"id":15612456,"url":"https://github.com/euberdeveloper/pastauctions-vavato-scraper","last_synced_at":"2025-10-13T17:31:50.916Z","repository":{"id":225191545,"uuid":"764879383","full_name":"euberdeveloper/pastauctions-vavato-scraper","owner":"euberdeveloper","description":null,"archived":false,"fork":false,"pushed_at":"2024-02-29T16:31:49.000Z","size":73,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-02-29T18:42:58.028Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/euberdeveloper.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2024-02-28T21:53:32.000Z","updated_at":"2024-02-29T18:43:10.682Z","dependencies_parsed_at":"2024-02-29T18:53:36.395Z","dependency_job_id":null,"html_url":"https://github.com/euberdeveloper/pastauctions-vavato-scraper","commit_stats":null,"previous_names":["euberdeveloper/pastauctions-vavato-scraper"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/euberdeveloper%2Fpastauctions-vavato-scraper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/euberdeveloper%2Fpastauctions-vavato-scraper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/euberdeveloper%2Fpastauctions-vavato-scraper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/euberdeveloper%2Fpastauctions-vavato-scraper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/euberdeveloper","download_url":"https://codeload.github.com/euberdeveloper/pastauctions-vavato-scraper/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":219870602,"owners_count":16555095,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-03T06:43:08.537Z","updated_at":"2025-10-13T17:31:45.587Z","avatar_url":"https://github.com/euberdeveloper.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pastauctions-vavato-scraper\nA web scraper to scrape the content of some car auctions from Vavato.\n\n## How to use it\n\nNotes: you will need python and pipenv installed in your system.\n\n1. Clone the repository\n2. Install the dependencies with `pipenv install`\n3. Run the script with `pipenv run python main.py`\n\nSome adjustemnts:\n- You should change the destination folder `save_path_prefix`\n- You can filter the \"categories\" of auctions from the variable `allowed_auctions_roots`\n- You can change the request delay in order to not be blocked because of too many requests by changing the variable `request_delay`\n- In case a block happens, the seconds before retrying can be changed in the variable `retry_delay`. At every retry it gets doubled.\n\n## What does it do\n\nThe script gets the auctions information and for each auction it gets the urls to the cars in the lots. Everything is divided into archived auctions and current/future actions. The result is an excel file with four sheets, one for the auctions and another for the car lots, for both archived and new auctions. \n\nIn `example_result` some example files are available.\n\n## More technical notes\n\nThe script uses normal http requests to navigate the website and get the information. This is much faster than using for example Selenium. The websites returns content that is already rendered and does not use AJAX to load the content. This makes it possible to get the content of the pages with requests.\n\nIn particular, each page has in the end a tag `\u003cscript type=\"application/json\"\u003e` that contains the information of the page. This is the information that is used to get the number of pages and the content for each page.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feuberdeveloper%2Fpastauctions-vavato-scraper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feuberdeveloper%2Fpastauctions-vavato-scraper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feuberdeveloper%2Fpastauctions-vavato-scraper/lists"}