{"id":19055166,"url":"https://github.com/sangupta/shopify-burst-crawler","last_synced_at":"2026-02-18T00:31:31.579Z","repository":{"id":39960243,"uuid":"109223985","full_name":"sangupta/shopify-burst-crawler","owner":"sangupta","description":"Simple crawler to download meta information for all stock pics from Shopify Burst website","archived":false,"fork":false,"pushed_at":"2022-09-01T22:34:46.000Z","size":49,"stargazers_count":2,"open_issues_count":3,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-24T04:17:52.371Z","etag":null,"topics":["burst","crawler","java","shopify","stock-photos"],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sangupta.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-11-02T06:03:18.000Z","updated_at":"2022-01-19T09:48:17.000Z","dependencies_parsed_at":"2022-08-25T09:10:14.250Z","dependency_job_id":null,"html_url":"https://github.com/sangupta/shopify-burst-crawler","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/sangupta/shopify-burst-crawler","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sangupta%2Fshopify-burst-crawler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sangupta%2Fshopify-burst-crawler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sangupta%2Fshopify-burst-crawler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sangupta%2Fshopify-burst-crawler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sangupta","download_url":"https://codeload.github.com/sangupta/shopify-burst-crawler/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sangupta%2Fshopify-burst-crawler/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29563467,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-17T21:50:49.831Z","status":"ssl_error","status_checked_at":"2026-02-17T21:46:15.313Z","response_time":100,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["burst","crawler","java","shopify","stock-photos"],"created_at":"2024-11-08T23:42:24.264Z","updated_at":"2026-02-18T00:31:31.561Z","avatar_url":"https://github.com/sangupta.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# shopify-burst-crawler\n\nSimple crawlers to download meta information for all stock photos from Shopify Burst \nwebsite: https://burst.shopify.com. There are 2 crawling modes supported:\n\n* BurstCrawler - starts crawling using the latest photos URL\n* BurstSitemapCrawler - starts crawling using the sitemap URL\n\n## Usage\n\nBoth crawlers support same API:\n\n```java\nBurstCrawler crawler = new BurstCrawler();\n\n// or you could use the Sitemap crawler\ncrawler = new BurstSitemapCrawler();\n\n// this will crawl and return the entire bunch of results\n// it is slow and may take a lot of time\nList\u003cBurstImage\u003e images = crawler.crawl();\n\n// a streaming version if also available\nGenericConsumer\u003cBurstImage\u003e collector = new GenericConsumer\u003cBurstImage\u003e() {\n\n\t@Override\n\tpublic boolean consume(BurstImage image) {\n\t\t// do something with the image\n\t\t// ...\n\t\t\n\t\t// return a true if you want to continue crawling\n\t\t// or return, a false to stop the crawling\n\t\treturn true;\n\t}\n};\ncrawler.crawl(collector);\n```\n\n## Downloads\n\nThe current development snapshot `JAR` can be obtained using `JitPack.io` as:\n\nAdd the following `repository` to Maven,\n\n```xml\n\u003crepository\u003e\n\t\u003cid\u003ejitpack.io\u003c/id\u003e\n\t\u003curl\u003ehttps://jitpack.io\u003c/url\u003e\n\u003c/repository\u003e\n```\n\nThen add the dependency as,\n\n```xml\n\u003cdependency\u003e\n    \u003cgroupId\u003ecom.github.sangupta\u003c/groupId\u003e\n    \u003cartifactId\u003eshopify-burst-crawler\u003c/artifactId\u003e\n    \u003cversion\u003e1.0.0-SNAPSHOT\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\n## Versioning\n\nFor transparency and insight into our release cycle, and for striving to maintain backward compatibility, \nthis project will be maintained under the Semantic Versioning guidelines as much as possible.\n\nReleases will be numbered with the follow format:\n\n`\u003cmajor\u003e.\u003cminor\u003e.\u003cpatch\u003e`\n\nAnd constructed with the following guidelines:\n\n* Breaking backward compatibility bumps the major\n* New additions without breaking backward compatibility bumps the minor\n* Bug fixes and misc changes bump the patch\n\nFor more information on SemVer, please visit http://semver.org/.\n\n## License\n\t\n```\nshopify-burst-crawler\nCopyright (c) 2017-2019, Sandeep Gupta\n\nhttps://sangupta.com/projects/shopify-burst-crawler\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n\thttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsangupta%2Fshopify-burst-crawler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsangupta%2Fshopify-burst-crawler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsangupta%2Fshopify-burst-crawler/lists"}