{"id":20165607,"url":"https://github.com/maxralph1/webscraper-python","last_synced_at":"2026-06-10T05:31:39.950Z","repository":{"id":186441053,"uuid":"675163350","full_name":"maxralph1/webscraper-python","owner":"maxralph1","description":null,"archived":false,"fork":false,"pushed_at":"2023-08-06T04:35:41.000Z","size":68,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-03T03:26:39.043Z","etag":null,"topics":["python","webscraper"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/maxralph1.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-06T02:08:01.000Z","updated_at":"2023-08-13T13:39:02.000Z","dependencies_parsed_at":null,"dependency_job_id":"4090d566-1f6e-4edd-986a-d339a1667d66","html_url":"https://github.com/maxralph1/webscraper-python","commit_stats":null,"previous_names":["maxralph1/webscraper-python"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/maxralph1/webscraper-python","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maxralph1%2Fwebscraper-python","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maxralph1%2Fwebscraper-python/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maxralph1%2Fwebscraper-python/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maxralph1%2Fwebscraper-python/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/maxralph1","download_url":"https://codeload.github.com/maxralph1/webscraper-python/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maxralph1%2Fwebscraper-python/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34139178,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-10T02:00:07.152Z","response_time":89,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["python","webscraper"],"created_at":"2024-11-14T00:38:26.208Z","updated_at":"2026-06-10T05:31:39.935Z","avatar_url":"https://github.com/maxralph1.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Webscraper Project\n\nThis is a web scraper project crafted in Python\n\n## Installation\n\n### Virtual Environment\n\nFirst, you would have to create a virtual environment if you would not want to install the required dependencies globally.\n\nUse the following command in the directory where you intend to house your application.\n\n```\npy -m venv venv\n```\n\nOR\n\n```\npython -m venv venv\n```\n\nThen, activate the virtual environment by running:\nOn Windows:\n\n```\n.\\venv\\Scripts\\activate\n```\n\nOR on Mac / GitBash:\n\n```\nsource ./venv/Scripts/activate\n```\n\n### Install the dependencies:\n\n```\npip install -r requirements.txt\n```\n\n### Optional: Global Installation of Dependencies\n\nYou may also wish to install globally, the dependencies we used. Thereby, skipping the virtual environment and requirements.txt file steps above.\nIf so, use these commands instead in any order:\n\n```\npip install beautifulsoup4\n```\n\n```\npip install lxml\n```\n\n```\npip install requests\n```\n\n## To run the webscraper:\n\nUse either of the following commands, based on your operating system.\n\n```\npy index.py\n```\n\nOR\n\n```\npython index.py\n```\n\n## To run the webscraper and have the results saved to a file:\n\nUse either of the following commands, based on your operating system.\n\n```\npy file_index.py\n```\n\nOR\n\n```\npython file_index.py\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaxralph1%2Fwebscraper-python","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmaxralph1%2Fwebscraper-python","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaxralph1%2Fwebscraper-python/lists"}