{"id":19662951,"url":"https://github.com/pustovitdmytro/parser","last_synced_at":"2025-08-03T18:33:23.268Z","repository":{"id":46230037,"uuid":"103038561","full_name":"pustovitDmytro/parser","owner":"pustovitDmytro","description":"tiny universal Python-based asynchronous sites scrapper","archived":false,"fork":false,"pushed_at":"2021-10-31T23:30:24.000Z","size":158,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-01-10T01:53:33.128Z","etag":null,"topics":["aiohttp","boilerplate","parser","python3","scrapper"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pustovitDmytro.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-09-10T15:05:45.000Z","updated_at":"2017-09-13T21:56:10.000Z","dependencies_parsed_at":"2022-08-31T02:41:30.493Z","dependency_job_id":null,"html_url":"https://github.com/pustovitDmytro/parser","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pustovitDmytro%2Fparser","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pustovitDmytro%2Fparser/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pustovitDmytro%2Fparser/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pustovitDmytro%2Fparser/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pustovitDmytro","download_url":"https://codeload.github.com/pustovitDmytro/parser/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240971427,"owners_count":19886745,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aiohttp","boilerplate","parser","python3","scrapper"],"created_at":"2024-11-11T16:12:58.430Z","updated_at":"2025-02-27T03:23:27.247Z","avatar_url":"https://github.com/pustovitDmytro.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# parser\ntiny universal Python-based asynchronous sites scrapper\n\n## Table of Contents\n1. [Requirements](#requirements)\n2. [Install \u0026 Run](#install-\u0026-run)\n3. [Project Structure](#project-structure)\n4. [Tests](#tests)\n5. [Related Projects](related-projects)\n6. [License](license)\n\n## Requirements\n* python `3.0+`\n* pip `9.0+`\n\n## Install \u0026 Run\n\nTo install packages run:\n```sh\n$ pip install bs4 selenium aiohttp asyncio async_timeout time abc re\n$ python script.py\n```\n\n## Project Structure\n\n```\n.\n├── html                            # folder with files for offline parsing\n│   └── google.example.page         # page with google request\n├── results                         # folder with parsed files\n├── screenshots                     # folder with selenium screenshots\n├── main.py                         # file with method implementation\n└── test.google.py                  # example of using\n```\n\n## Tests\n\nBest way to test web-parser, is to parse some site with data templates. For instance in `test.google.py` you can explore a way of parsing google search pages.\nRun it by\n```\n$ python test.google.py\n```\n\n## Related Projects\n\nThere are some  more boilerplates that I'm working on:\n1. [gulp](https://github.com/pustovitDmytro/gulp) - simple template for quick site creation via gulp\n2. [web-extension](https://github.com/pustovitDmytro/web-extension) - boilerplate to making web-extensions (firefox and chrome simultaneously)\n3. [react](https://github.com/pustovitDmytro/react) - boilerplate for making gracefull react apps\n\n## Examples Of Use\n\nThere are some examples of implementation:\n* [dmsu-bot](https://github.com/pustovitDmytro/dmsu-bot) - bot which helped me to order biometric passport\n* [quotes](https://github.com/pustovitDmytro/quotes) - scraps famous quotes and saves them to scv\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpustovitdmytro%2Fparser","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpustovitdmytro%2Fparser","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpustovitdmytro%2Fparser/lists"}