{"id":31566231,"url":"https://github.com/a-nau/easy-image-scraping","last_synced_at":"2025-10-05T07:27:20.653Z","repository":{"id":54054291,"uuid":"521975416","full_name":"a-nau/easy-image-scraping","owner":"a-nau","description":"Web application to automatically scrape images from Google, Bing, Baidu and Yahoo.","archived":false,"fork":false,"pushed_at":"2023-08-04T00:30:33.000Z","size":4849,"stargazers_count":7,"open_issues_count":1,"forks_count":3,"subscribers_count":3,"default_branch":"main","last_synced_at":"2023-08-04T01:49:58.541Z","etag":null,"topics":["baidu","bing","docker","frontend","google-images","gui","image-scraper","image-scrapping","web-application","yahoo"],"latest_commit_sha":null,"homepage":"https://a-nau.github.io/parcel2d/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/a-nau.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-08-06T14:55:05.000Z","updated_at":"2023-08-04T01:49:58.542Z","dependencies_parsed_at":"2023-02-16T20:01:23.782Z","dependency_job_id":null,"html_url":"https://github.com/a-nau/easy-image-scraping","commit_stats":null,"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"purl":"pkg:github/a-nau/easy-image-scraping","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/a-nau%2Feasy-image-scraping","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/a-nau%2Feasy-image-scraping/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/a-nau%2Feasy-image-scraping/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/a-nau%2Feasy-image-scraping/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/a-nau","download_url":"https://codeload.github.com/a-nau/easy-image-scraping/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/a-nau%2Feasy-image-scraping/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":278424888,"owners_count":25984679,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-05T02:00:06.059Z","response_time":54,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["baidu","bing","docker","frontend","google-images","gui","image-scraper","image-scrapping","web-application","yahoo"],"created_at":"2025-10-05T07:27:19.387Z","updated_at":"2025-10-05T07:27:20.645Z","avatar_url":"https://github.com/a-nau.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![arxiv](http://img.shields.io/badge/paper-arxiv.2210.09814-B31B1B.svg)][arxiv]\n[![project page](https://img.shields.io/badge/website-project%20page-informational.svg)][project page]\n\n# Easy Image Scraping from Google, Bing, Yahoo and Baidu\n\nAutomatically scrape images with your query from the popular search engines\n\n- Google\n- Bing\n- Baidu\n- Yahoo (currently only low resolution)\n\nusing an easy-to-use Frontend or using scripts.\n\n![](data/misc/demo.gif)\n\nThis code is part of a [paper][arxiv] ([citation](#citation)), also check\nthe [project page][project page] if you are interested in creation a dataset for instance segmentation.\n\n## Usage\n\n### Front End\n\nStart the front end with a single command (adjust the `/PATH/TO/OUTPUT` to your desired output path)\n\n```shell\ndocker run -it --rm --name easy_image_scraping --mount type=bind,source=/PATH/TO/OUTPUT,target=/usr/src/app/output -p 5000:5000 ghcr.io/a-nau/easy-image-scraping:latest\n```\n\nEnter your query and wait for the results to show in the `output` folder. The web applications also shows a preview of\ndownloaded images.\n\n### Command Line\n\nStart using the command line with\n\n```shell\ndocker run -it --rm --name easy_image_scraping --mount type=bind,source=/PATH/TO/OUTPUT,target=/usr/src/app/output -p 5000:5000 ghcr.io/a-nau/easy-image-scraping:latest bash\n```\n\n#### Search for a keyword\n\nIf you just want to search for a single keywords adjust and run [`search_by_keyword.py`](src/tools/search_by_keyword.py)\n\n#### Search for a list of keywords\n\n- Write the list of search terms in the file `search_terms_eng.txt`.\n- You can then use [Google Translate](https://translate.google.com/) to translate the whole file to new languages.\n  Change the ending of the translated file to the respective language.\n- Adjust [`config.py`](src/config.py) to define search engines for each language\n- Run [`search_by_keywords_from_files`](src/tools/search_by_keywords_from_files.py)\n\n## Installation (optional)\n\nThis is optional - you can also directly use our provided container.\n\n### Docker\n\nYou can also build the image yourself using\n\n```shell\ndocker build -t easy_image_scraping .\n```\nThe run it by using\n```shell\ndocker run -it --rm --name easy_image_scraping -p 5000:5000 --mount type=bind,source=/PATH/TO/OUTPUT,target=/usr/src/app/output easy_image_scraping\n```\n\n\u003cdetails\u003e\n  \u003csummary\u003eFor Local Setup, check this\u003c/summary\u003e\n\n### Local installation\n\n- Set up an environment using\n  ```shell\n  conda env create -f environment.yml\n  ```\n  or\n  ```shell\n  pip install -r requirements.txt\n  ```\n- To use [Selenium](https://www.selenium.dev/), we need to download\n  the [Chrome Driver](https://sites.google.com/chromium.org/driver/) (also\n  see [this](https://www.selenium.dev/documentation/webdriver/getting_started/install_drivers/))\n- [Check](https://www.google.com/intl/us/chrome/update/) your Chrome Version and download the corresponding webdriver\n  version\n- Unzip it, and add it to the path (for details, see [here](https://stackoverflow.com/a/40556092)). Alternatively, you\n  can adjust [scrape_and_download.py](src/scraping/scrape_and_download.py)\n  ```python\n  with webdriver.Chrome(\n      executable_path=\"path/to/chrome_diver.exe\",  # add this line\n      options=set_chrome_options()\n  ) as wd:\n  ```\n\n\u003c/details\u003e\n\n## Affiliations\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\"https://upload.wikimedia.org/wikipedia/de/thumb/4/44/Fzi_logo.svg/1200px-Fzi_logo.svg.png?raw=true\" alt=\"FZI Logo\" height=\"200\"/\u003e\n\u003c/p\u003e\n\n## License and Credits\n\n- Code is partially based on and borrowed from\n  - [sczhengyabin/Image-Downloader](https://github.com/sczhengyabin/Image-Downloader) (\n    mostly [crawler.py](https://github.com/sczhengyabin/Image-Downloader/blob/master/crawler.py))\n    , [MIT License](https://github.com/sczhengyabin/Image-Downloader/blob/master/LICENSE)\n  - [Article](https://towardsdatascience.com/image-scraping-with-python-a96feda8af2d) with Gists\n    by [Fabian Bosler](https://medium.com/@fabianbosler), see [fetch_image_urls.py](src/scraping/fetch_image_urls.py)\n- Dockerfile is based\n  on [joyzoursky/ docker-python-chromedriver](https://github.com/joyzoursky/docker-python-chromedriver/blob/master/py-debian/3.9-selenium/Dockerfile)\n  , [MIT License](https://github.com/joyzoursky/docker-python-chromedriver/blob/master/LICENSE)\n- Cookie notices are handled by the [I still don't care about cookies](https://github.com/OhMyGuus/I-Still-Dont-Care-About-Cookies) extension [GNU General Public License v3.0](http://172.21.127.85:5000/)\n\nUnless stated otherwise, this project is licensed under the [MIT](LICENSE) license.\n\n## Citation\n\nIf you use this code for scientific research, please consider citing\n\n```latex\n@inproceedings{naumannScrapeCutPasteLearn2022,\n\ttitle        = {Scrape, Cut, Paste and Learn: Automated Dataset Generation Applied to Parcel Logistics},\n\tauthor       = {Naumann, Alexander and Hertlein, Felix and Zhou, Benchun and Dörr, Laura and Furmans, Kai},\n\tbooktitle    = {{{IEEE Conference}} on {{Machine Learning}} and Applications ({{ICMLA}})},\n\tdate         = 2022\n}\n```\n\n## Disclaimer\n\nPlease be aware of copyright restrictions that might apply to images you download.\n\n[arxiv]: https://arxiv.org/abs/2210.09814\n[project page]: https://a-nau.github.io/parcel2d\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fa-nau%2Feasy-image-scraping","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fa-nau%2Feasy-image-scraping","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fa-nau%2Feasy-image-scraping/lists"}