{"id":13737551,"url":"https://github.com/ahmedbesbes/dataset-builder","last_synced_at":"2025-07-14T18:31:12.388Z","repository":{"id":50733235,"uuid":"211855366","full_name":"ahmedbesbes/dataset-builder","owner":"ahmedbesbes","description":"A script to help you quickly build custom computer vision datasets","archived":false,"fork":false,"pushed_at":"2020-03-02T12:56:45.000Z","size":15203,"stargazers_count":34,"open_issues_count":0,"forks_count":6,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-11-15T05:32:54.726Z","etag":null,"topics":["classification","computer-vision","dataset-creation","detection","segmentation","selenium"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ahmedbesbes.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-09-30T12:32:42.000Z","updated_at":"2024-11-07T19:21:30.000Z","dependencies_parsed_at":"2022-09-03T07:02:16.623Z","dependency_job_id":null,"html_url":"https://github.com/ahmedbesbes/dataset-builder","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahmedbesbes%2Fdataset-builder","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahmedbesbes%2Fdataset-builder/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahmedbesbes%2Fdataset-builder/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ahmedbesbes%2Fdataset-builder/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ahmedbesbes","download_url":"https://codeload.github.com/ahmedbesbes/dataset-builder/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225990427,"owners_count":17556155,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["classification","computer-vision","dataset-creation","detection","segmentation","selenium"],"created_at":"2024-08-03T03:01:53.009Z","updated_at":"2024-11-23T02:16:37.404Z","avatar_url":"https://github.com/ahmedbesbes.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Dataset Builder\n\nA script to help you quickly build custom computer vision datasets for classification and detection.\n\nIt allows you to define your classes and then fetches images from Flickr.\nIt then organizes the folders and clean the file names.\nIf you're preparing the dataset for a detection or a segmentation task, the script opens up makesense.ai, uploads the images with the corresponding label list so that you can start annotating.\n\nOnce the annotation is done, your labels can be exported and you'll be ready to train your awesome models.\n\n### Requirements:\n\n- Install Flick Scraper dependencies:\n  - `git clone https://github.com/ultralytics/flickr_scraper`\n  - `cd flickr_scraper`\n  - `pip install -U -r requirements.txt`\n  - Request a Flickr API key and secret: https://www.flickr.com/services/apps/create/apply\n  - create a `config.yaml` file inside `src/` that looks like this\n    ```yaml\n    key: \"XXXXXXXXXXXXXXX\"\n    secret: \"XXXXXXXXXXXXXXX\"\n    ```\n- Selenium: pip install -U selenium\n- ChromeDriver 77.0.3865.40\n\nIn case you wish to run Maskesense locally:\n\n```bash\n# clone repository\ngit clone https://github.com/SkalskiP/make-sense.git\n\n# navigate to main dir\ncd make-sense\n\n# install dependencies\nnpm install\n\n# serve with hot reload at localhost:3000\nnpm start\n```\n\n### Example:\n\nWhen you run the script, you can specify the following arguments:\n\n- `output-directory`: the root folder when images are downloaded\n- `limit`: the maximum number of downloaded images per category\n- `delete-history`: whether you choose to erase previous downloads or not\n- `task`: classification, detection or segmentation\n- `driver`: path to chrome driver\n\n```bash\npython dataset_builder.py --limit 20 --delete-history yes\n```\n\nOnce the script runs, you'll be asked to define your classes (or queries)\n\n\u003cimg src = \"./images/screenshot.png\" \u003e\n\nHere's what the output looks like after the download:\n\n\u003cimg src = \"./images/output.png\" width=350\u003e\n\n### Object detection with make-sense:\n\nThis only works if you choose a **detection** or **segmentation** task\n\nMake Sense is an awesome open source webapp that lets you easily label your image dataset for tasks such as localization.\n\nYou can check it out here: https://www.makesense.ai/\nYou can also clone it and run it locally (for better performance): https://github.com/SkalskiP/make-sense\n\nIn order to use this tool, I'll be running it locally and interface with it using Selenium: Once the dataset is downloaded, Selenium opens up a Chrome browser, upload the images to the app and fill in the label list: this ultimately allows you to annotate.\n\n### Demo [Youtube video]\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://www.youtube.com/watch?v=qXLvMr9mrP4\"\u003e\n    \u003cimg src=\"https://img.youtube.com/vi/qXLvMr9mrP4/0.jpg\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n### TODO :grin:\n\nPlease feel free to contribute ! Report any bugs in the issue section, or request any feature you'd like to see shipped:\n\n- [ ] Accelerate the download of images via multiprocessing\n- [ ] Apply a quality check on the images\n- [ ] Integrate automatic tagging using pre-trained networks\n\n### LICENSE\n\nPlease be aware that this code is under the GPL3 license.\nYou must report each utilisation of this code to the author of this code (ahmedbesbes).\nPlease push your code using this API on a forked Github repo public.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fahmedbesbes%2Fdataset-builder","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fahmedbesbes%2Fdataset-builder","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fahmedbesbes%2Fdataset-builder/lists"}