{"id":13771646,"url":"https://github.com/bcmi/Foreground-Object-Search-Dataset-FOSD","last_synced_at":"2025-05-11T04:30:48.303Z","repository":{"id":186018999,"uuid":"667686982","full_name":"bcmi/Foreground-Object-Search-Dataset-FOSD","owner":"bcmi","description":"[ICCV 2023] The datasets and code used in our paper \"Foreground Object Search by Distilling Composite Image Feature\", ICCV2023. ","archived":false,"fork":false,"pushed_at":"2025-01-19T03:30:28.000Z","size":47,"stargazers_count":20,"open_issues_count":0,"forks_count":1,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-01-19T04:26:13.513Z","etag":null,"topics":["foreground-object-search","image-composition"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bcmi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2023-07-18T04:54:01.000Z","updated_at":"2025-01-19T03:30:30.000Z","dependencies_parsed_at":"2023-11-10T14:45:10.244Z","dependency_job_id":"8b353b1a-6679-40bf-b006-f7476ecb4946","html_url":"https://github.com/bcmi/Foreground-Object-Search-Dataset-FOSD","commit_stats":null,"previous_names":["bcmi/foreground-object-search-dataset-fosd"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bcmi%2FForeground-Object-Search-Dataset-FOSD","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bcmi%2FForeground-Object-Search-Dataset-FOSD/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bcmi%2FForeground-Object-Search-Dataset-FOSD/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bcmi%2FForeground-Object-Search-Dataset-FOSD/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bcmi","download_url":"https://codeload.github.com/bcmi/Foreground-Object-Search-Dataset-FOSD/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253518941,"owners_count":21921074,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["foreground-object-search","image-composition"],"created_at":"2024-08-03T17:00:53.903Z","updated_at":"2025-05-11T04:30:48.286Z","avatar_url":"https://github.com/bcmi.png","language":"Python","funding_links":[],"categories":["Datasets"],"sub_categories":[],"readme":"# Foreground Object Search Dataset FOSD\n\nThis is the official repository for the following paper:\n\n\u003e **Foreground Object Search by Distilling Composite Image Feature**  [[arXiv]](https://arxiv.org/pdf/2308.04990.pdf)\u003cbr\u003e\n\u003e\n\u003e Bo Zhang, Jiacheng Sui, Li Niu\u003cbr\u003e\n\u003e Accepted by **ICCV 2023**.\n\n**Our model has been integrated into our image composition toolbox libcom https://github.com/bcmi/libcom. Welcome to visit and try ＼(^▽^)／** \n\n## Requirements\n\n- See requirements.txt for other dependencies.\n\n## Data Preparing\n\n- Download Open-Images-v6 trainset from [Open Images V6 - Download](https://storage.googleapis.com/openimages/web/download_v6.html) and unzip them. We recommend that you use FiftyOne to download the Open-Images-v6 dataset. After the dataset is downloaded, the data structure of Open-Images-v6 dataset should be as follows.\n  \n  ```\n  Open-Images-v6\n  ├── metadata\n  ├── train\n  │   ├── data\n  │   │   ├── xxx.jpg\n  │   │   ├── xxx.jpg\n  │   │   ...\n  │   │\n  │   └── labels\n  │       └── masks\n  │       │   ├── 0\n  │       │       ├── xxx.png\n  │       │       ├── xxx.png\n  │       │       ...\n  │       │   ├── 1\n  │       │   ...\n  │       │\n  │       ├── segmentations.csv\n  │       ...\n  ```\n\n- Download S-FOSD annotations, R-FOSD annotations and background images of R-FOSD from [Baidu disk](https://pan.baidu.com/s/1LF_4LbwxbxSBy-zqBkgzDw) (code: 3wvf) and save them to the appropriate location under the `data` directory according to the data structure below. \n  \n- Generate backgrounds and foregrounds.\n  \n  ```\n  python prepare_data/fetch_data.py --open_images_dir \u003cpath/to/open/images\u003e\n  ```\n\nThe data structure is like this:\n\n```\ndata\n├── metadata\n│   ├── classes.csv\n│   └── category_embeddings.pkl\n├── test\n│   ├── bg_set1\n│   │   ├── xxx.jpg\n│   │   ├── xxx.jpg\n│   │   ...\n│   │\n│   ├── bg_set2\n│   │   ├── xxx.jpg\n│   │   ├── xxx.jpg\n│   │   ...\n│   │\n│   ├── fg\n│   │   ├── xxx.jpg\n│   │   ├── xxx.jpg\n│   │   ...\n│   └── labels\n│       └── masks\n│       │   ├── 0\n│       │       ├── xxx.png\n│       │       ├── xxx.png\n│       │       ...\n│       │   ├── 1\n│       │   ...\n│       │\n│       ├── test_set1.json\n│       ├── test_set2.json\n│       └── segmentations.csv\n│\n└── train\n    ├── bg\n    │   ├── xxx.jpg\n    │   ├── xxx.jpg\n    │   ...\n    │\n    ├── fg\n    │   ├── xxx.jpg\n    │   ├── xxx.jpg\n    │   ...\n    │\n    └── labels\n        └── masks\n        │   ├── 0\n        │       ├── xxx.png\n        │       ├── xxx.png\n        │       ...\n        │   ├── 1\n        │   ...\n        │\n        ├── train_sfosd.json\n        ├── train_rfosd.json\n        ├── category.json\n        ├── number_per_category.csv\n        └── segmentations.csv\n```\n\n## Pretrained Model\n\nWe provide the checkpoint ([Baidu disk](https://pan.baidu.com/s/1_Dh2w08AAqdsw8Cb3l4nfQ) code: 7793) for the evaluation on S-FOSD dataset and checkpoint ([Baidu disk](https://pan.baidu.com/s/17jq1FWKSsEngp7scB4357Q) code: 6kme) for testing on R-FOSD dataset. By default, we assume that the pretrained model is downloaded and saved to the directory `checkpoints`.\n\n## Testing\n\n### Evaluation on S-FOSD Dataset\n\n```\npython evaluate/evaluate.py --testOnSet1\n```\n\n### Evaluation on R-FOSD Dataset\n\n```\npython evaluate/evaluate.py --testOnSet2\n```\n\nThe evaluation results will be stored to the directory `eval_results`.\n\nIf you want to save top 20 results on R-FOSD, add `--saveTop20 ` parameter. The top 20 results on R-FOSD will be stored to the directory `top20` by default.\n\nIf you want to save the model's prediction scores on R-FOSD, add `--saveScores` parameter. The model scores on R-FOSD will be stored to the directory `model_scores` by default.\n\n## Training\n\nPlease download the pretrained teacher models from [Baidu disk](https://pan.baidu.com/s/1D_zT326PLXZ-C0j5mcCY6A) (code: 40a5) and save the model to directory `checkpoints/teacher`. \n\nTo train a new sfosd model, you can simply run:\n\n```\n.train/train_sfosd.sh\n```\n\nSimilarly, train a new rfosd model by:\n\n```\n.train/train_rfosd.sh\n```\n\n## FOS Score\n\nOur model can be used to evaluate the compatibility between foreground and background in terms of geometry and semantics.\n\nTo launch the demo, you can run:\n\n```\npython demo/demo_ui.py\n```\n\nHere are three steps you can take to get a compatibility  score for the foreground and the background.\n\n1) Upload a background image in the left box of the first row\n\n2) Click the left-top point and the right-bottom point of the bounding box in the right box of the first row\n\n3) Upload a foreground image in the left box of the second row, then click 'run' button.\n\n## Other Resources\n\n+ [Awesome-Foreground-Object-Search](https://github.com/bcmi/Awesome-Foreground-Object-Search)\n+ [Awesome-Image-Composition](https://github.com/bcmi/Awesome-Object-Insertion)\n\n## License\n\nBoth background and foreground images of S-FOSD belong to Open-Images. The background images of R-FOSD are collected from Internet and are licensed under a Creative Commons Attribution 4.0 License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbcmi%2FForeground-Object-Search-Dataset-FOSD","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbcmi%2FForeground-Object-Search-Dataset-FOSD","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbcmi%2FForeground-Object-Search-Dataset-FOSD/lists"}