{"id":19425633,"url":"https://github.com/rese1f/uniap","last_synced_at":"2025-07-10T11:09:33.899Z","repository":{"id":185932748,"uuid":"674338709","full_name":"rese1f/UniAP","owner":"rese1f","description":"[AAAI 2024] UniAP: Towards Universal Animal Perception in Vision via Few-shot Learning","archived":false,"fork":false,"pushed_at":"2023-12-10T21:46:35.000Z","size":1201,"stargazers_count":12,"open_issues_count":0,"forks_count":3,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-06-21T09:07:34.098Z","etag":null,"topics":["2d-pose-estimation","animal","classification","computer-vision","semantic-segmentation"],"latest_commit_sha":null,"homepage":"https://rese1f.github.io/UniAP/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rese1f.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-08-03T17:45:36.000Z","updated_at":"2025-01-11T16:21:53.000Z","dependencies_parsed_at":"2023-12-20T13:24:41.558Z","dependency_job_id":"fbba886a-1c4e-4abe-bb5c-7765467f1f4a","html_url":"https://github.com/rese1f/UniAP","commit_stats":{"total_commits":17,"total_committers":3,"mean_commits":5.666666666666667,"dds":0.2941176470588235,"last_synced_commit":"78d7ee802351e3ca52b115b2c773638e98e65caa"},"previous_names":["rese1f/uniap"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/rese1f/UniAP","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rese1f%2FUniAP","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rese1f%2FUniAP/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rese1f%2FUniAP/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rese1f%2FUniAP/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rese1f","download_url":"https://codeload.github.com/rese1f/UniAP/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rese1f%2FUniAP/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261095308,"owners_count":23108784,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["2d-pose-estimation","animal","classification","computer-vision","semantic-segmentation"],"created_at":"2024-11-10T14:04:25.724Z","updated_at":"2025-06-21T09:07:39.684Z","avatar_url":"https://github.com/rese1f.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg src=\"assets/uniap_logo.png\" height=\"120px\" align=\"right\"\u003e\n\n# UniAP\n\n[![](http://img.shields.io/badge/cs.CV-arXiv%3A2308.09953-B31B1B.svg)](https://arxiv.org/abs/2308.09953)\n\n\u003e **UniAP: Towards Universal Animal Perception in Vision via Few-shot Learning**  \n\u003e Meiqi Sun*, Zhonghan Zhao*, Wenhao Chai*, Hanjun Luo, Shidong Cao, Yanting Zhang, Jenq-Neng Hwang, Gaoang Wang   \n\u003e _AAAI 2024_  \n\nWe introduce UniAP, a novel Universal Animal Perception model that leverages few-shot learning to enable cross-species perception among various visual tasks.\n\n## :fire: News\n* **[2023.12.10]**: 🎉 Our paper is accepted by AAAI 2024.\n* **[2023.08.20]** : We release our code.\n* **[2023.08.19]** :page_with_curl: We release the [paper](https://arxiv.org/abs/https://arxiv.org/abs/2308.09953).\n\n\u003ch3 align=\"center\"\u003e If you like our project, please give us a star ⭐ on GitHub for the latest update.\u003c/h3\u003e\n\n## Setup\n1. Download Datasets\n  * Animal Kingdom Dataset (pose estimation) from the official GitHub page https://github.com/sutdcv/Animal-Kingdom/blob/master/Animal_Kingdom/pose_estimation/README_pose_estimation.md.\n  * Animal Pose Dataset from the official GitHub page https://github.com/noahcao/animal-pose-dataset.\n  * APT-36K Dataset from the official GitHub page https://github.com/pandorgan/APT-36K.\n  * Oxford-IIIT Pet Dataset from the official page https://www.robots.ox.ac.uk/~vgg/pets/.\n  * (Optional) Resize the images and labels into (256, 256) resolution.\n  * We stored data from all animal images and labels in a single directory. The directory structure looks like:\n  ```\n  \u003cRoot\u003e\n  |--\u003cAnimalKingdom\u003e\n  |   |--\u003canimal1\u003e_\u003crgb\u003e\n  |   | ...\n  |   |--\u003canimal2\u003e_\u003clabel\u003e\n  |   |...\n  |\n  |--\u003cAPT-36K\u003e\n  |   |--\u003canimal1\u003e_\u003crgb\u003e\n  |   | ...\n  |   |--\u003canimal2\u003e_\u003clabel\u003e\n  |   |...\n  |\n  |--\u003cAnimalPose\u003e\n  |   |--\u003canimal1\u003e_\u003crgb\u003e\n  |   | ...\n  |   |--\u003canimal2\u003e_\u003clabel\u003e\n  |   |...\n  |\n  |--\u003cOxford-IIITPet\u003e\n  |   |--\u003canimal1\u003e_\u003crgb\u003e\n  |   | ...\n  |   |--\u003canimal2\u003e_\u003clabel\u003e\n  |   |...\n  |\n  |...\n  ```\n\n1. Create `data_paths.yaml` file and write the root directory path (`\u003cRoot\u003e` in the above structure) by `UniASET: PATH_TO_YOUR_UniASET`.\n\n2. Install pre-requirements by `pip install -r requirements.txt`.\n\n3. Create `model/pretrained_checkpoints` directory and download [BEiT pre-trained checkpoints](https://github.com/microsoft/unilm/tree/master/beit) to the directory.\n  * We used `beit_base_patch16_224_pt22k` checkpoint for our experiment.\n  * We also provided the pre-trained model trained on the AnimalKingdom dataset that can be used to run the configs/demo.yaml (https://drive.google.com/file/d/1HmSMn1h4rY5JtEjS7Th8iPTJhFbAnW9x/view?usp=sharing)\n\n## Usage\n\n### Training\n```\npython main.py --stage 0 --task_id [0/1/2/3]\n```\n  * If you want to train universally on all tasks, please set `task_id=3`. \n  * If you want to train on the specific task, please follow `task_id=0`: pose estimation, `task_id=1`: semantic segmentation, `task_id=2`: classification.\n### Fine-tuning\n\n```\npython main.py --stage 1 --task [kp/mask/cls]\n```\n* If you want to finetune on the specific task, please follow `task=kp`: pose estimation, `task=mask`: semantic segmentation, `task=cls`: classification.\n### Evaluation\n\n```\npython main.py --stage 2 --task [kp/mask/cls]\n```\n* If you want to evaluate on the specific task, please follow `task=kp`: pose estimation, `task=mask`: semantic segmentation, `task=cls`: classification.\n\n## Acknowledgements\nOur code refers the following repositores:\n* [BEiT: BERT Pre-Training of Image Transformers](https://github.com/microsoft/unilm/tree/master/beit)\n* [Pose for Everything: Towards Category-Agnostic Pose Estimation](https://github.com/luminxu/Pose-for-Everything)\n* [Images Speak in Images: A Generalist Painter for In-Context Visual Learning](https://github.com/baaivision/Painter)\n* [Segment Anything](https://github.com/facebookresearch/segment-anything)\n* [Contrastive Language-Image Pre-Training](https://github.com/openai/CLIP)\n\n## Citation\n\nIf you find STEVE useful for your your research and applications, please cite using this BibTeX:\n\n```bibtex\n@article{sun2023uniap,\n  title={UniAP: Towards Universal Animal Perception in Vision via Few-shot Learning},\n  author={Sun, Meiqi and Zhao, Zhonghan and Chai, Wenhao and Luo, Hanjun and Cao, Shidong and Zhang, Yanting and Hwang, Jenq-Neng and Wang, Gaoang},\n  journal={arXiv preprint arXiv:2308.09953},\n  year={2023}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frese1f%2Funiap","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frese1f%2Funiap","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frese1f%2Funiap/lists"}