{"id":20715680,"url":"https://github.com/ovshake/cobra","last_synced_at":"2026-03-07T05:05:29.055Z","repository":{"id":40961393,"uuid":"262855904","full_name":"ovshake/cobra","owner":"ovshake","description":"Code for COBRA: Contrastive Bi-Modal Representation Algorithm (https://arxiv.org/abs/2005.03687)","archived":false,"fork":false,"pushed_at":"2023-07-06T21:54:54.000Z","size":488,"stargazers_count":15,"open_issues_count":5,"forks_count":3,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-29T23:22:56.829Z","etag":null,"topics":["contrastive-learning","cross-modal","machine-learning","pytorch","representation-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ovshake.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-05-10T19:02:28.000Z","updated_at":"2024-06-21T03:06:52.000Z","dependencies_parsed_at":"2022-09-12T03:21:48.008Z","dependency_job_id":null,"html_url":"https://github.com/ovshake/cobra","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ovshake%2Fcobra","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ovshake%2Fcobra/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ovshake%2Fcobra/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ovshake%2Fcobra/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ovshake","download_url":"https://codeload.github.com/ovshake/cobra/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250419441,"owners_count":21427602,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["contrastive-learning","cross-modal","machine-learning","pytorch","representation-learning"],"created_at":"2024-11-17T02:39:31.886Z","updated_at":"2026-03-07T05:05:29.023Z","avatar_url":"https://github.com/ovshake.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# COBRA: Contrastive Bi-Modal Representation Algorithm\n\nThis repository contains the code for the paper COBRA: Contrastive Bi-Modal Representation Algorithm [(ArXiv)](https://arxiv.org/abs/2005.03687) by [Vishaal Udandarao](https://vishaal27.github.io/), [Abhishek Maiti](http://ovshake.me), Deepak Srivatsav, Suryatej Reddy, Yifang Yin and [Rajiv Ratn Shah](http://faculty.iiitd.ac.in/~rajivratn/). \n\n## Methodology \nWe present a novel framework COBRA that aims to train two modalities (image and text) in a joint fashion inspired by the **Contrastive Predictive Coding (CPC)** and **Noise Contrastive Estimation (NCE)** paradigms which preserve both inter and intra-class relationships. We empirically show that this framework reducesthe modality gap significantly and generates a robust and task agnostic joint-embedding space. We outperform existing work on four diverse downstream tasks spanning across seven benchmark cross-modal datasets.\u003cbr\u003e\n\n**A visualisation of the loss function**:\n\u003cbr\u003e\n\u003cbr\u003e\n![Supervised Contrastive Loss](https://github.com/ovshake/cobra/blob/master/images/CPCLoss.JPG)\n\n\n\n## Architecture\n\n![](https://github.com/ovshake/cobra/blob/master/images/Architecture.JPG)\n\n\n## Datasets\nThe 7 datasets used to empirically prove our results are:\u003cbr\u003e \n1. PKU-XMedia \n2. MS-COCO \n3. NUS-Wide 10k \n4. Wikipedia\n5. FakeNewsNet\n6. MeTooMA\n7. CrisisMMD\n\n## t-SNEs\n![](https://github.com/ovshake/cobra/blob/master/images/t-SNE_1.JPG)\n![](https://github.com/ovshake/cobra/blob/master/images/t-SNE_2.JPG)\n## Results\n\n![Wikipedia](https://github.com/ovshake/cobra/blob/master/images/Wiki.JPG)\n![MS-COCO](https://github.com/ovshake/cobra/blob/master/images/MSCoco.JPG)\n![PKU-XMedia](https://github.com/ovshake/cobra/blob/master/images/PKU-XMedia.JPG)\n![MS-COCO](https://github.com/ovshake/cobra/blob/master/images/NUS-Wide.JPG)\n![MS-COCO](https://github.com/ovshake/cobra/blob/master/images/FakeNewsNet.JPG)\n![MS-COCO](https://github.com/ovshake/cobra/blob/master/images/MeTooMA.JPG)\n![MS-COCO](https://github.com/ovshake/cobra/blob/master/images/CrisisMMD.JPG)\n\n\n## Instructions for running\nThe code has been tested on Python 3.6.8 and PyTorch 1.5.1. \n- Install all the dependencies using the following command:\n```\npip install -r requirements.txt\n```\n- Create a folder `features` to save the trained models\n- To train COBRA, use the following command:\n```\npython main.py\n```\n- To switch between NCE contrastive loss and softmax contrastive loss, change the `use_nce` flag. To change the number of anchor points and number of negative samples, modify the `num_anchors` and `num_negative_samples` respectively.\n## Queries\nIn case of any queries, please open an issue. We will respond as soon as possible. \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fovshake%2Fcobra","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fovshake%2Fcobra","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fovshake%2Fcobra/lists"}