{"id":13526160,"url":"https://github.com/kimhc6028/relational-networks","last_synced_at":"2025-04-01T06:31:19.910Z","repository":{"id":21678314,"uuid":"93711918","full_name":"kimhc6028/relational-networks","owner":"kimhc6028","description":"Pytorch implementation of \"A simple neural network module for relational reasoning\" (Relational Networks)","archived":false,"fork":false,"pushed_at":"2022-12-06T04:12:56.000Z","size":59303,"stargazers_count":811,"open_issues_count":9,"forks_count":160,"subscribers_count":25,"default_branch":"master","last_synced_at":"2024-11-02T10:34:12.300Z","etag":null,"topics":["deep-learning","pytorch","relational-networks"],"latest_commit_sha":null,"homepage":"https://arxiv.org/pdf/1706.01427.pdf","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kimhc6028.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"License.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-06-08T05:48:41.000Z","updated_at":"2024-09-19T03:16:39.000Z","dependencies_parsed_at":"2023-01-11T21:18:53.116Z","dependency_job_id":null,"html_url":"https://github.com/kimhc6028/relational-networks","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kimhc6028%2Frelational-networks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kimhc6028%2Frelational-networks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kimhc6028%2Frelational-networks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kimhc6028%2Frelational-networks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kimhc6028","download_url":"https://codeload.github.com/kimhc6028/relational-networks/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246596888,"owners_count":20802917,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","pytorch","relational-networks"],"created_at":"2024-08-01T06:01:25.965Z","updated_at":"2025-04-01T06:31:18.779Z","avatar_url":"https://github.com/kimhc6028.png","language":"Python","funding_links":[],"categories":["Python","Paper implementations｜论文实现","Paper implementations","Paper Implementations"],"sub_categories":["Other libraries｜其他库:","Other libraries:"],"readme":"Pytorch implementation of Relational Networks - [A simple neural network module for relational reasoning](https://arxiv.org/pdf/1706.01427.pdf)\n\nImplemented \u0026 tested on Sort-of-CLEVR task.\n\n## Sort-of-CLEVR\n\nSort-of-CLEVR is simplified version of [CLEVR](http://cs.stanford.edu/people/jcjohns/clevr/).This is composed of 10000 images and 20 questions (10 relational questions and 10 non-relational questions) per each image. 6 colors (red, green, blue, orange, gray, yellow) are assigned to randomly chosen shape (square or circle), and placed in a image.\n\nNon-relational questions are composed of 3 subtypes:\n\n1) Shape of certain colored object\n2) Horizontal location of certain colored object : whether it is on the left side of the image or right side of the image\n3) Vertical location of certain colored object : whether it is on the upside of the image or downside of the image\n\nTheses questions are \"non-relational\" because the agent only need to focus on certain object.\n\nRelational questions are composed of 3 subtypes:\n\n1) Shape of the object which is closest to the certain colored object\n1) Shape of the object which is furthest to the certain colored object\n3) Number of objects which have the same shape with the certain colored object\n\nThese questions are \"relational\" because the agent has to consider the relations between objects.\n\nQuestions are encoded into a vector of size of 11 : 6 for one-hot vector for certain color among 6 colors, 2 for one-hot vector of relational/non-relational questions. 3 for one-hot vector of 3 subtypes.\n\n\u003cimg src=\"./data/sample.png\" width=\"256\"\u003e\n\nI.e., with the sample image shown, we can generate non-relational questions like:\n\n1) What is the shape of the red object? =\u003e Circle (even though it does not really look like \"circle\"...)\n2) Is green object placed on the left side of the image? =\u003e yes\n3) Is orange object placed on the upside of the image? =\u003e no\n\nAnd relational questions:\n\n1) What is the shape of the object closest to the red object? =\u003e square\n2) What is the shape of the object furthest to the orange object? =\u003e circle\n3) How many objects have same shape with the blue object? =\u003e 3\n\n## Setup\n\nCreate conda environment from `environment.yml` file\n```\n$ conda env create -f environment.yml\n```\nActivate environment\n```\n$ conda activate RN3\n```\nIf you don't use conda install python 3 normally and use `pip install` to install remaining dependencies. The list of dependencies can be found in the `environment.yml` file.\n\n## Usage\n\n\t$ ./run.sh\n\nor\n\n  \t$ python sort_of_clevr_generator.py\n\nto generate sort-of-clevr dataset\nand\n\n \t $ python main.py \n\nto train the binary RN model. \nAlternatively, use \n\n \t $ python main.py --relation-type=ternary\n\nto train the ternary RN model.\n\n## Modifications\nIn the original paper, Sort-of-CLEVR task used different model from CLEVR task. However, because model used CLEVR requires much less time to compute (network is much smaller), this model is used for Sort-of-CLEVR task.\n\n## Result\n\n| | Relational Networks (20th epoch) | CNN + MLP (without RN, 100th epoch) |\n| --- | --- | --- |\n| Non-relational question | 99% | 66% |\n| Relational question | 89% | 66% |\n\nCNN + MLP occured overfitting to the training data.\n\nRelational networks shows far better results in relational questions and non-relation questions. \n\n## Contributions\n\n[@gngdb](https://github.com/gngdb) speeds up the model by 10 times.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkimhc6028%2Frelational-networks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkimhc6028%2Frelational-networks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkimhc6028%2Frelational-networks/lists"}