{"id":18658272,"url":"https://github.com/ruofeidu/ducrawler","last_synced_at":"2025-04-11T19:32:16.539Z","repository":{"id":67601882,"uuid":"114024849","full_name":"ruofeidu/DuCrawler","owner":"ruofeidu","description":"An automatic crawler to mine images from Google and Bing Image search (part of SketchyScene at ECCV 2018)","archived":false,"fork":false,"pushed_at":"2022-03-08T02:03:29.000Z","size":34,"stargazers_count":12,"open_issues_count":1,"forks_count":7,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-25T17:47:40.672Z","etag":null,"topics":["data","google","image","mining","python","search"],"latest_commit_sha":null,"homepage":"https://sketchyscene.github.io/SketchyScene","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ruofeidu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-12-12T18:25:29.000Z","updated_at":"2023-02-24T02:47:38.000Z","dependencies_parsed_at":null,"dependency_job_id":"fe02f591-0077-4686-ba66-215c919879bd","html_url":"https://github.com/ruofeidu/DuCrawler","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ruofeidu%2FDuCrawler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ruofeidu%2FDuCrawler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ruofeidu%2FDuCrawler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ruofeidu%2FDuCrawler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ruofeidu","download_url":"https://codeload.github.com/ruofeidu/DuCrawler/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248467026,"owners_count":21108587,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data","google","image","mining","python","search"],"created_at":"2024-11-07T07:32:15.127Z","updated_at":"2025-04-11T19:32:16.531Z","avatar_url":"https://github.com/ruofeidu.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# DuCrawler\n\nMy crawler to mine image from Google and Bing image search\n\n## Dependencies of crawler_google\n\n* Python 2.7 / 3.6, compatiable with python 3.0+\n* pip install bs4\n* pip install requests\n* pip install opencv-contrib-python\n* pip2.7 install configparser\n\n## Additional dependencies of crawler_bing\n\n// pip install -U selenium\n\n* pip install selenium==2.48.0\n* see [Selenium](https://pypi.python.org/pypi/selenium)\n* [PhantomJS 2.1.1](http://phantomjs.org/download.html)\n* pip3.6 install urllib\n* or pip2.7 install urlparse\n\n## Author\n\n[Ruofei Du](http://duruofei.com)\n\n## References\n\n[Writing Python 2-3 compatible code](http://python-future.org/compatible_idioms.html#unicode)\n\n## License\n\nCreative Commons Attribution-NonCommercial-ShareAlike 3.0 License with 996 ICU clause: [![996.ICU](https://img.shields.io/badge/link-996.icu-red.svg)](https://996.icu/#/en_US)\n\nThe above license is only granted to entities that act in concordance with local labor laws. In addition, the following requirements must be observed:\n\n* The licensee must not, explicitly or implicitly, request or schedule their employees to work more than 45 hours in any single week.\n* The licensee must not, explicitly or implicitly, request or schedule their employees to be at work consecutively for 10 hours.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fruofeidu%2Fducrawler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fruofeidu%2Fducrawler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fruofeidu%2Fducrawler/lists"}