{"id":19282286,"url":"https://github.com/jaybfn/shopify_webscraping","last_synced_at":"2025-08-23T06:06:20.653Z","repository":{"id":138331648,"uuid":"457280146","full_name":"jaybfn/Shopify_WebScraping","owner":"jaybfn","description":null,"archived":false,"fork":false,"pushed_at":"2022-03-01T12:57:16.000Z","size":375,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-23T06:06:19.263Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jaybfn.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-02-09T08:55:11.000Z","updated_at":"2022-02-09T08:57:01.000Z","dependencies_parsed_at":null,"dependency_job_id":"c2f2af4f-38af-445f-8fa8-9a5b8d271cad","html_url":"https://github.com/jaybfn/Shopify_WebScraping","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jaybfn/Shopify_WebScraping","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaybfn%2FShopify_WebScraping","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaybfn%2FShopify_WebScraping/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaybfn%2FShopify_WebScraping/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaybfn%2FShopify_WebScraping/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jaybfn","download_url":"https://codeload.github.com/jaybfn/Shopify_WebScraping/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaybfn%2FShopify_WebScraping/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271745692,"owners_count":24813516,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-23T02:00:09.327Z","response_time":69,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T21:26:03.606Z","updated_at":"2025-08-23T06:06:20.588Z","avatar_url":"https://github.com/jaybfn.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Shopify_WebScraping\n\nWeb scraping (sometimes called web data extraction or data scraping) is a technique that allows users to automatically acquire structured web data. Web scraping is a useful alternative if the public website you want to get data from doesn't have an API, or if it has but just allows you limited access to the data.\n\n**This code is only used for shopify based websites\n\n![alt picture](https://github.com/jaybfn/Shopify_WebScraping/blob/main/web_scrapping.jpg)\nimage credit source https://www.loginworks.com/blogs/web-scraping-breakthrough-data-harvesting/\n\n### Scraping the web is a multi-step process:\n\n1. Go to the webpage you want to scrape and make an HTTP request.\n2. The target webpage's HTML content is returned by the server in response to the request.\n3. Parse the HTML content in step two.\n4. To generate a layered structure of HTML data, you'll need a parser.\n5. Extract data from HTML\n6. To extract and save data, we use Python libraries like Json and Beautiful Soup and database libraries.\n\n### Tech requirements:\n1. requests\n2. json\n3. dataset\n4. logging\n5. python 3.9 \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjaybfn%2Fshopify_webscraping","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjaybfn%2Fshopify_webscraping","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjaybfn%2Fshopify_webscraping/lists"}