{"id":16233961,"url":"https://github.com/ashutosh1919/context-cluster-pytorch","last_synced_at":"2025-04-03T11:30:35.983Z","repository":{"id":159891030,"uuid":"620054615","full_name":"ashutosh1919/context-cluster-pytorch","owner":"ashutosh1919","description":"Ready to run PyTorch implementation of Context-Cluster: Image as Set of Points","archived":false,"fork":false,"pushed_at":"2023-05-01T14:44:00.000Z","size":22,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-25T18:22:52.851Z","etag":null,"topics":["computer-vision","deep-learning","image-classification","machine-learning","pytorch","segmentation","segmentation-models"],"latest_commit_sha":null,"homepage":"https://blog.paperspace.com/context-cluster/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ashutosh1919.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-03-28T00:04:48.000Z","updated_at":"2023-08-03T07:50:34.000Z","dependencies_parsed_at":null,"dependency_job_id":"c6405fb5-8448-4262-a273-22042122a1e3","html_url":"https://github.com/ashutosh1919/context-cluster-pytorch","commit_stats":{"total_commits":7,"total_committers":1,"mean_commits":7.0,"dds":0.0,"last_synced_commit":"aef3e91b9b96cf5b14f5d6a11fc9e75491d00bd9"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashutosh1919%2Fcontext-cluster-pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashutosh1919%2Fcontext-cluster-pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashutosh1919%2Fcontext-cluster-pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ashutosh1919%2Fcontext-cluster-pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ashutosh1919","download_url":"https://codeload.github.com/ashutosh1919/context-cluster-pytorch/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246963460,"owners_count":20861486,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","deep-learning","image-classification","machine-learning","pytorch","segmentation","segmentation-models"],"created_at":"2024-10-10T13:14:23.456Z","updated_at":"2025-04-03T11:30:35.948Z","avatar_url":"https://github.com/ashutosh1919.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Context-Cluster\n\nContext-Cluster is entirely new paradigm that considers image as Set of Points and generates representations. This repository contains ready-to train models \u0026 visualize clusters over images. This repository utilizes original [Context-Cluster](https://github.com/ma-xu/Context-Cluster) ([arxiv](https://arxiv.org/abs/2303.01494)) implementation. Moreover, we have created helper scripts to train the models and Gradio app to generate cluster visualization for any image.  \n\nIf you want to understand Context-Cluster in detail, check out [this blog on Paperspace](https://blog.paperspace.com/context-cluster/).\n\n\n## Run in a Free GPU powered Gradient Notebook\n[![Gradient](https://assets.paperspace.io/img/gradient-badge.svg)](https://console.paperspace.com/github/ashutosh1919/context-cluster-pytorch?machine=Free-GPU)\n\n\n## Setup\n\nThe file `installations.sh` contains all the necessary code to install required things. Note that your system must have CUDA to train Context-Cluster models. Also, you may require different version of `torch` based on the version of CUDA. If you are running this on [Paperspace](https://www.paperspace.com/), then the default version of CUDA is 11.6 which is compatible with this code. If you are running it somewhere else, please check your CUDA version using `nvcc --version`. If the version differs from ours, you may want to change versions of PyTorch libraries in the first line of `installations.sh` by looking at [compatibility table](https://github.com/pytorch/pytorch/wiki/PyTorch-Versions).\n\nTo install all the dependencies, run below command:\n\n```bash\nbash installations.sh\n```\n\nAbove command also clones the original [Context-Cluster](https://github.com/ma-xu/Context-Cluster) repository into `context_cluster` directory so that we can utilize the original model implementation for training \u0026 inference.\n\n\n## Downloading datasets \u0026 Start training (Optional)\n\n`dataset` directory in this repo contains necessary scripts to download the data and make it ready for training. Currently, this repository supports downloading [ImageNet](https://www.image-net.org/) dataset that original authors used.\n\nWe have already setup bash scripts for you which will automatically download the dataset for you and will start the training. `train.sh` contains the code which will download the training \u0026 validation data to `dataset` directory and will start training the model.\n\nThis bash script is compatible to the Paperspace workspace. But if you are running it elsewhere, then you will need to replace base path of the paths mentioned in this script `train.sh`.\n\nBefore you start the training, you can check \u0026 customize all the model arguments in `args.yaml` file. Especially, you may want to change the argument `model` to one of the following: `coc_tiny`, `coc_tiny_plain`, `coc_small`, `coc_medium`. These models differ by the number of layers (stages).\n\nTo download data files and start training, you can execute below command:\n\n```bash\nbash train.sh\n```\n\nNote that the generated checkpoints for the trained model will be available in `context_cluster/outputs` directory. You will need to move `checkpoint.pth.tar` file to `checkpoints` directory for inference at the end of training.\n\nDon't worry if you don't want to train the model. Below section illustrates downloading the pretrained checkpoints for inference.\n\n\n## Running Gradio Demo\n\nPython script `app.py` contains Gradio demo which lets you visualize clusters on the image. But before we do that, we need to download the pretrained checkpoints into `checkpoints` directory.\n\nTo download existing checkpoints, run below command:\n\n```bash\nbash checkpoints/fetch_pretrained_checkpoints.sh\n```\n\nNote that the latest version of code only has the pretrained checkpoints for `coc_tiny_plain` model type. But you can add the code in `fetch_pretrained_checkpoints.sh` whenever the new checkpoints for other model types are available in [original repository](https://github.com/ma-xu/Context-Cluster).\n\nNow, we are ready to launch Gradio demo. Run following command to launch demo:\n\n```bash\ngradio app.py\n```\n\nOpen the link in the browser and now you can generate inference from any of available models in `checkpoints` directory. Moreover, you can generate cluster visualization of specific `stage`, `block` and `head`. Upload your image and hit the Submit button.\n\nYou should be able to generate cluster visualization for any image as shown below:\n\n\u003cimg width=\"1238\" alt=\"demo\" src=\"https://user-images.githubusercontent.com/20843596/228104200-aed8b4dc-ebce-4d41-b097-34d63fe7f993.png\"\u003e\n\nHurray! 🎉🎉🎉  We have created demo to visualize clusters over any image by inferring Context-Cluster model.\n\n\n## Original Code\n\n`context_cluster` directory contains the original code taken from [Context-Cluster](https://github.com/ma-xu/Context-Cluster) repository. The code present in this directory is exactly same as the original code.\n\n\n## Reference\n\nImage as Set of Points -- https://arxiv.org/abs/2303.01494\n\n```\n@misc{ma2023image,\n      title={Image as Set of Points}, \n      author={Xu Ma and Yuqian Zhou and Huan Wang and Can Qin and Bin Sun and Chang Liu and Yun Fu},\n      year={2023},\n      eprint={2303.01494},\n      archivePrefix={arXiv},\n      primaryClass={cs.CV}\n}\n```\n\n## License\n\nSee the [LICENSE](LICENSE) file.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fashutosh1919%2Fcontext-cluster-pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fashutosh1919%2Fcontext-cluster-pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fashutosh1919%2Fcontext-cluster-pytorch/lists"}