{"id":17458159,"url":"https://github.com/pocesar/actor-shopify-scraper","last_synced_at":"2025-07-06T04:33:11.641Z","repository":{"id":43318012,"uuid":"386804921","full_name":"pocesar/actor-shopify-scraper","owner":"pocesar","description":"Automate monitoring prices on the most popular solution for building online stores and selling products online. Crawl arbitrary Shopify-powered online stores and extract a list of all products in a structured form, including product title, price, description, etc","archived":false,"fork":false,"pushed_at":"2023-06-05T17:12:50.000Z","size":37,"stargazers_count":13,"open_issues_count":2,"forks_count":11,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-05-25T07:59:57.464Z","etag":null,"topics":["apify","apify-sdk","javascript","scraper","scraping","shopify"],"latest_commit_sha":null,"homepage":"https://apify.com/pocesar/shopify-scraper","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pocesar.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-07-17T01:18:26.000Z","updated_at":"2024-08-09T20:30:26.000Z","dependencies_parsed_at":"2024-10-20T19:23:17.179Z","dependency_job_id":null,"html_url":"https://github.com/pocesar/actor-shopify-scraper","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/pocesar/actor-shopify-scraper","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pocesar%2Factor-shopify-scraper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pocesar%2Factor-shopify-scraper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pocesar%2Factor-shopify-scraper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pocesar%2Factor-shopify-scraper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pocesar","download_url":"https://codeload.github.com/pocesar/actor-shopify-scraper/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pocesar%2Factor-shopify-scraper/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263848974,"owners_count":23519552,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apify","apify-sdk","javascript","scraper","scraping","shopify"],"created_at":"2024-10-18T03:55:44.709Z","updated_at":"2025-07-06T04:33:11.574Z","avatar_url":"https://github.com/pocesar.png","language":"JavaScript","readme":"## What does Shopify Scraper do?\n\nUsing this tool, you can automate monitoring prices on the most popular solution for building online stores and selling products online. Crawl arbitrary Shopify-powered online stores and extract a list of all products in a structured form, including product title, price, description, etc.\n\n## Need to find product pairs between Shopify and another online shop?\n\nUse the [AI Product Matcher](https://apify.com/equidem/ai-product-matcher). This AI model allows you to compare items from different web stores, identifying exact matches and comparing real-time data obtained via web scraping. With the AI Product Matcher, you can use scraped product data to monitor product matches across the industry, implement dynamic pricing for your website, replace or complement manual mapping, and obtain realistic estimates against your competition for upcoming promo campaigns. \n\nMost importantly, it is relatively easy to get started with (just follow [this guide](https://blog.apify.com/product-matching-ai-pricing-intelligence-web-scraping/)) and it can match thousands of product pairs.\n\n## Extend Scraper and Output Function\n\nExtend output function allows to filter the items that are output:\n\n```js\nasync ({ item, customData }) =\u003e {\n    if (!item.title.includes('cuisine')) {\n        return null; // omit the output\n    }\n\n    delete item.additional; // remove data from output\n\n    item.requestId = customData.requestId; // add data from the outside\n\n    return item;\n}\n```\n\nExtend scraper function allows you to interact with scraper phases:\n\n```js\nasync ({ label, url, filter, fns, filteredSitemapUrls, customData }) =\u003e {\n    switch (label) {\n        case 'FILTER_SITEMAP_URL': {\n            // product url, like .../products/cooking-for-dummies-2002-289854\n            filter(\n                url.includes('cooking') || url.includes(customData.filter)\n            );\n            break;\n        }\n        case 'SETUP': {\n            // filteredSitemapUrls is a `Set` instance and can be edited in-place\n            filteredSitemapUrls.add('https://example.com/secret-unlisted-sitemap.xml');\n            filteredSitemapUrls.forEach((sitemapURL) =\u003e {\n                if (!sitemapURL.includes('en-us')) {\n                    filteredSitemapUrls.delete(sitemapURL);\n                }\n            });\n            break;\n        }\n    }\n}\n```\n\n## License\n\nApache 2.0\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpocesar%2Factor-shopify-scraper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpocesar%2Factor-shopify-scraper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpocesar%2Factor-shopify-scraper/lists"}