{"id":16096336,"url":"https://github.com/sio/scrapehelper","last_synced_at":"2025-07-17T23:33:57.929Z","repository":{"id":101312962,"uuid":"191552464","full_name":"sio/scrapehelper","owner":"sio","description":"Helpful library for scraping information from web","archived":false,"fork":false,"pushed_at":"2023-05-05T13:25:03.000Z","size":19,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-06-18T01:44:33.626Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-06-12T10:49:28.000Z","updated_at":"2023-05-05T13:25:07.000Z","dependencies_parsed_at":"2023-06-19T16:01:47.583Z","dependency_job_id":null,"html_url":"https://github.com/sio/scrapehelper","commit_stats":{"total_commits":19,"total_committers":1,"mean_commits":19.0,"dds":0.0,"last_synced_commit":"fa0db6a06b57f60b2b1b2033d0949ef07c3ec2c6"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sio/scrapehelper","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sio%2Fscrapehelper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sio%2Fscrapehelper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sio%2Fscrapehelper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sio%2Fscrapehelper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sio","download_url":"https://codeload.github.com/sio/scrapehelper/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sio%2Fscrapehelper/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265678642,"owners_count":23810115,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-09T17:13:29.567Z","updated_at":"2025-07-17T23:33:57.894Z","avatar_url":"https://github.com/sio.png","language":"Python","readme":"# Helpful library for scraping information from web\n\n## Project status\n\nNew project, used by one person. The API changes should be backwards\ncompatible most of the time.\n\n\n## Overview\n\nThis library provides helper tools for efficient and polite web scraping:\n\n- Thread safe `RateLimiter` object\n- Nice `BaseDataFetcher` class for creating custom data fetchers\n\n\n## Installation\n\n1. As a standalone Python package:\n`pip install \"https://github.com/sio/scrapehelper/tarball/master\"`\n\n2. As a dependency in your setup.py:\n```python\n    install_requires=[\n        'scrapehelper @ https://github.com/sio/scrapehelper/tarball/master',\n        # other dependencies\n    ],\n```\n\n\n## Usage\n\nAPI docs are yet to be written. The primary objects provided by this library\nare `scrapehelper.fetch.BaseDataFetcher` and `scrapehelper.limit.RateLimiter`.\n\nCheck the code of the corresponding modules for more information. Submitting\ndocumentation improvements via pull requests is very welcome!\n\n\n## Support and contributing\n\nIf you need help with including this library into your Python project, please\ncreate **[an issue](https://github.com/sio/scrapehelper/issues)**. Issues are\nalso the primary venue for reporting bugs and posting feature requests.\nGeneral discussion related to this project is also acceptable and very\nwelcome!\n\nIn case you wish to contribute code or documentation, feel free to open **[a\npull request](https://github.com/sio/scrapehelper/pulls)**. That would certainly\nmake my day!\n\nI'm open to dialog and I promise to behave responsibly and treat all\ncontributors with respect. Please try to do the same, and treat others the way\nyou want to be treated.\n\nIf for some reason you'd rather not use the issue tracker, contacting me via\nemail is OK too. Please use a descriptive subject line to enhance visibility\nof your message. Also please keep in mind that public discussion channels are\npreferable because that way many other people may benefit from reading past\nconversations. My email is visible under the GitHub profile and in the commit\nlog.\n\n\n\n## License and copyright\n\nCopyright 2019 Vitaly Potyarkin\n\n    Licensed under the Apache License, Version 2.0 (the \"License\");\n    you may not use this file except in compliance with the License.\n    You may obtain a copy of the License at\n\n        http://www.apache.org/licenses/LICENSE-2.0\n\n    Unless required by applicable law or agreed to in writing, software\n    distributed under the License is distributed on an \"AS IS\" BASIS,\n    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n    See the License for the specific language governing permissions and\n    limitations under the License.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsio%2Fscrapehelper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsio%2Fscrapehelper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsio%2Fscrapehelper/lists"}