{"id":24409144,"url":"https://github.com/vtalpaert/pytorch-geometric-visual-task","last_synced_at":"2026-04-27T09:01:57.061Z","repository":{"id":97511837,"uuid":"236787495","full_name":"vtalpaert/pytorch-geometric-visual-task","owner":"vtalpaert","description":"Simple task for mixed image-graph data","archived":false,"fork":false,"pushed_at":"2020-01-28T16:59:23.000Z","size":905,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-12-27T19:21:16.518Z","etag":null,"topics":["dataset","deep-learning-datasets","geometric-deep-learning","graph-neural-networks","machine-learning-dataset","pytorch","pytorch-geometric"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vtalpaert.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-01-28T16:55:42.000Z","updated_at":"2021-06-18T20:41:58.000Z","dependencies_parsed_at":null,"dependency_job_id":"ba1afea1-7fc8-411b-9a91-60f0407b7e90","html_url":"https://github.com/vtalpaert/pytorch-geometric-visual-task","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/vtalpaert/pytorch-geometric-visual-task","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vtalpaert%2Fpytorch-geometric-visual-task","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vtalpaert%2Fpytorch-geometric-visual-task/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vtalpaert%2Fpytorch-geometric-visual-task/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vtalpaert%2Fpytorch-geometric-visual-task/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vtalpaert","download_url":"https://codeload.github.com/vtalpaert/pytorch-geometric-visual-task/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vtalpaert%2Fpytorch-geometric-visual-task/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32329466,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-26T23:26:28.701Z","status":"online","status_checked_at":"2026-04-27T02:00:06.769Z","response_time":128,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dataset","deep-learning-datasets","geometric-deep-learning","graph-neural-networks","machine-learning-dataset","pytorch","pytorch-geometric"],"created_at":"2025-01-20T05:54:42.804Z","updated_at":"2026-04-27T09:01:57.052Z","avatar_url":"https://github.com/vtalpaert.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PyTorch Geometric Visual Task: find the n-th closest node\n\nWe solve a task mixing a visual input with message passing between nodes.\n\nWhile we can define many different tasks from this repository's code, we choose a simple one: \"Who is the n-th closest node to a visual clue?\"\n\n(keywords: torch_geometric, pytorch, torch, graphconv, graph, dataset, python, machine_learning, deep_learning)\n\n## Dataset and task\n\nThis project is a simple on the fly dataset of a simple task : in a random graph with cartesian coordinates for node features, find the n-th closest node to a colored square on a random image.\n\nSee the example below:\n\n![output no radius](img/radius0_epoch190.png)\n\nWe use the PyTorch Geometric framework. Our `Data` object provides the background image tensor, the node features and edge indexes (COO format) and the groundtruth in a \"one hot\" vector or simply the index.\n\nSince we work with 0 indexed lists, finding the first closest node is Task-0\n\nAn example while learning to find the 2nd closest point (Task-1) :\n\n![Training example](img/training_example.png)\n\nWe provide several functions for plotting results in `draw.py`. The color intensity from white to the target color (red) is the network output score.\n\n## Results\n\nWe choose to fix some parameters for the dataset. While the learning rate can be adapted to the Task, we found these values work most of the time.\n\n```python\n    IM_CHANNELS = 3\n    IM_HEIGHT = IM_WIDTH = 64  # proximity to the center is calculated in normalized space, so beware to use only square images for now\n    NUM_NODES = 20\n    RADIUS = 0.3\n    MAX_NUM_NEIGHBORS = 3\n    TARGET_COLOR = (1.,0.,0.)  # red\n    BATCH_SIZE = 64\n    AUXILIARY_TASK_WEIGHT = 3\n    NUM_WORKERS = 8\n    SIZE_TRAIN = 8000\n    SIZE_TEST = 2000\n    EPOCHS = 200\n    LEARNING_RATE = 0.003  # 0.005 does well in Task-0 but unstable in Task-1\n    SCHEDULER_STEP_SIZE = 20\n    SCHEDULER_GAMMA = 0.75\n```\n\nOur network extracts features from the image (with a convnet similar to AlexNet), and concatenate the `image_features` vector to the node features. For this experiment we used an auxiliary task to force the image feature to be the square cartesian position (plotted with the black dot). We chose image feature extraction first to prepare for future experiences.\n\nThe baseline is a MLP outputing a score for each node looking only at the node and image feature pair.\nIn Task-0, the baseline reaches around 90% accuracy as it learns the distance function.\nFor Task-1, the baseline learns a bad distance function in the hope of being right sometimes, but goes down to 20% accuracy.\n\nRandom response gets 5% accuracy (`1/NUM_NODES`).\n\n### Task-0 with 20 nodes\n\nResults for an ealy version to benchmark different modules provided by PyTorch Geometric:\n\n| Method | Acc 10e (*) | Acc 50e (*) | Acc 80e (*) | Acc 200e (1) |\n| :------------- | :----------: | :-----------: | :-----------: | :-----------: |\n|  Linear (baseline) | . | . | . | ~90\n|  GCN max_i/max_i | 19.8 | 23.2 | 23.9 |\n|  GCN mean/mean (2) | 21.4 | 28.6 | 30.0 |\n|  GCN max_i/mean | 22.6 | 28.0 | 28.0 |\n|  GCN mean_i/max_i | 17.1 | 24.7 | 23.8 |\n|  GCN add/add | 5.8 | 31.2 | 32.6 |\n|  SAGE_yn (3) | 32.3 | 32.9 | 32.5 | 33.5 |\n|  SAGE_nn | 31.3 | 30.9 | 32.0 | 32.6\n|  [REDO]Graph max/max | (61) | (67) | (66) |\n|  Graph mean/mean | 55.0 | 66.6 | 69.0 | 69.8 |\n|  Graph mean/max | . | . | . | . |\n|  Graph add/add | 54.4 | 62.9 | 67.3 | 68.3 |\n|  GAT h1c1 (4) | 30.6 | 33.8 | 33.3 | 34.3 |\n|  [REDO]GAT h1c0 | (31) | (35) | (36) | (33.6)\n|  TAG K1 (5) | 54.0 | 63.9 | 66.7 | 67.4 |\n|  TAG K3 | 53.3 | 63.7 | 66.3 | 68.1\n|  [REDO]TAG K9 | 5.5 | 51.9 | 59.6 | 65.8\n|  SG K1 | 30.9 | 33.1 | 32.3 | 33.6\n|  SG K3 | 21.9 | 20.1 | 21.9 | 21.0\n\n- (*) Accuracy at X epochs, taken with 0.5 smoothing on tensorboard\n- (1) For the values at 200 epochs, we use 0.9 smoothing\n- (2) `max`/`mean`/`add` are different aggregation schemes, but `mean` provides generally better results\n- (3) \"SAGE y(es)n(o)\" means SAGE conv with normalization first layer, but not second\n- (4) GAT is h number of heads, c contenation or not\n- (5) K is the number of hops for TAG or SG convs\n\n## Final results\n\nThe baseline is two consecutive linear layers of sizes (4, 8), (8, 1):\n\n![Baseline results](img/baseline_training.png)\n\nFor the [GraphConv](https://pytorch-geometric.readthedocs.io/en/latest/modules/nn.html#torch_geometric.nn.conv.GraphConv), we use 8 successive layers of size 4 (expect final size of 1):\n\n![GraphConv results](img/graphconv_training.png)\n\nFor the [TAGConv](https://pytorch-geometric.readthedocs.io/en/latest/modules/nn.html#torch_geometric.nn.conv.TAGConv), we use 5 successive layers with K=3 and size 4:\n\n![TAGConv results](img/tagconv_training.png)\n\nWe use LeakyReLU as the activation unit. Hyperparameters are chosen according for stability and performance.\n\n## Training stability\n\nWe have occasional loss divergence since due to the sigmoid, sometimes a method gets \"stuck\" with the image_features equal to the extremes 0 or 1. To avoid this, we use a custom tanh which does not constrain the image_features between [0, 1] strictly. Could create a viz problem.\nWe also detach() the tensor since we are more experimenting with the message passing. But our goal was to get rid of the auxilliary loss so this is temporary.\nAdding noise to the image features could be interesting.\nWe could benefit from replacing the softmax with a log_softmax and adapting the loss function.\n\n## TODOs\n\n- Create directly a batch: it is faster now to create the data on CPU, batch it then send it to GPU than creating each data point on GPU then batch. One dev could be creating a batch directly, but not a priority.\n- Random target color: the auxilliary task is not our concern, it would be more visually pleasing but of no interest here\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvtalpaert%2Fpytorch-geometric-visual-task","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvtalpaert%2Fpytorch-geometric-visual-task","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvtalpaert%2Fpytorch-geometric-visual-task/lists"}