{"id":14958287,"url":"https://github.com/benedekrozemberczki/attentionwalk","last_synced_at":"2025-04-07T10:22:40.389Z","repository":{"id":48915139,"uuid":"165262775","full_name":"benedekrozemberczki/AttentionWalk","owner":"benedekrozemberczki","description":"A PyTorch Implementation of \"Watch Your Step: Learning Node Embeddings via Graph Attention\" (NeurIPS 2018).","archived":false,"fork":false,"pushed_at":"2022-11-06T21:09:53.000Z","size":1307,"stargazers_count":325,"open_issues_count":0,"forks_count":48,"subscribers_count":8,"default_branch":"master","last_synced_at":"2025-03-31T08:12:03.214Z","etag":null,"topics":["attention","deep-learning","deepwalk","graph-attention","graph-neural-networks","graph-representation-learning","implicit-factorization","machine-learning","matrix-factorization","neurips","neurips-2018","nips","node2vec","pytorch","sklearn","structural-attention","tensorflow","torch","walklet","word2vec"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/benedekrozemberczki.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null},"funding":{"github":["benedekrozemberczki"]}},"created_at":"2019-01-11T15:09:12.000Z","updated_at":"2025-03-23T13:30:46.000Z","dependencies_parsed_at":"2023-01-22T09:45:44.904Z","dependency_job_id":null,"html_url":"https://github.com/benedekrozemberczki/AttentionWalk","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benedekrozemberczki%2FAttentionWalk","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benedekrozemberczki%2FAttentionWalk/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benedekrozemberczki%2FAttentionWalk/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/benedekrozemberczki%2FAttentionWalk/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/benedekrozemberczki","download_url":"https://codeload.github.com/benedekrozemberczki/AttentionWalk/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247632454,"owners_count":20970164,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["attention","deep-learning","deepwalk","graph-attention","graph-neural-networks","graph-representation-learning","implicit-factorization","machine-learning","matrix-factorization","neurips","neurips-2018","nips","node2vec","pytorch","sklearn","structural-attention","tensorflow","torch","walklet","word2vec"],"created_at":"2024-09-24T13:16:40.784Z","updated_at":"2025-04-07T10:22:40.370Z","avatar_url":"https://github.com/benedekrozemberczki.png","language":"Python","funding_links":["https://github.com/sponsors/benedekrozemberczki"],"categories":[],"sub_categories":[],"readme":"Attention Walk\n=====================\n[![Arxiv](https://img.shields.io/badge/ArXiv-1710.09599-orange.svg?color=blue)](https://arxiv.org/abs/1710.09599) [![codebeat badge](https://codebeat.co/badges/adfe22f2-9122-4f58-953c-11d70b976e0e)](https://codebeat.co/projects/github-com-benedekrozemberczki-attentionwalk-master) [![repo size](https://img.shields.io/github/repo-size/benedekrozemberczki/AttentionWalk.svg)](https://github.com/benedekrozemberczki/AttentionWalk/archive/master.zip) [![benedekrozemberczki](https://img.shields.io/twitter/follow/benrozemberczki?style=social\u0026logo=twitter)](https://twitter.com/intent/follow?screen_name=benrozemberczki)\n\nA **PyTorch** Implementation of **Watch Your Step: Learning Node Embeddings via Graph Attention (NIPS 2018)**.\n\u003cdiv style=\"text-align:center\"\u003e\u003cimg src =\"attentionwalk.jpg\" ,width=720/\u003e\u003c/div\u003e\n\n### Abstract \n\u003cp align=\"justify\"\u003e\nGraph embedding methods represent nodes in a continuous vector space, preserving different types of relational information from the graph. There are many hyper-parameters to these methods (e.g. the length of a random walk) which have to be manually tuned for every graph. In this paper, we replace previously fixed hyper-parameters with trainable ones that we automatically learn via backpropagation. In particular, we propose a novel attention model on the power series of the transition matrix, which guides the random walk to optimize an upstream objective. Unlike previous approaches to attention models, the method that we propose utilizes attention parameters exclusively on the data itself (e.g. on the random walk), and are not used by the model for inference. We experiment on link prediction tasks, as we aim to produce embeddings that best-preserve the graph structure, generalizing to unseen information. We improve state-of-the-art results on a comprehensive suite of real-world graph datasets including social, collaboration, and biological networks, where we observe that our graph attention model can reduce the error by up to 20%-40%. We show that our automatically-learned attention parameters can vary significantly per graph, and correspond to the optimal choice of hyper-parameter if we manually tune existing methods.\u003c/p\u003e\n\nThis repository provides an implementation of Attention Walk as described in the paper:\n\n\u003e Watch Your Step: Learning Node Embeddings via Graph Attention.\n\u003e Sami Abu-El-Haija, Bryan Perozzi, Rami Al-Rfou, Alexander A. Alemi.\n\u003e NIPS, 2018.\n\u003e [[Paper]](http://papers.nips.cc/paper/8131-watch-your-step-learning-node-embeddings-via-graph-attention)\n\nThe original Tensorflow implementation is available [[here]](https://github.com/google-research/google-research/tree/master/graph_embedding/watch_your_step).\n\n### Requirements\n\nThe codebase is implemented in Python 3.5.2. package versions used for development are just below.\n```\nnetworkx          2.4\ntqdm              4.28.1\nnumpy             1.15.4\npandas            0.23.4\ntexttable         1.5.0\nscipy             1.1.0\nargparse          1.1.0\ntorch             1.1.0\ntorchvision       0.3.0\n```\n### Datasets\n\u003cp align=\"justify\"\u003e\nThe code takes an input graph in a csv file. Every row indicates an edge between two nodes separated by a comma. The first row is a header. Nodes should be indexed starting with 0. Sample graphs for the `Twitch Brasilians` and `Wikipedia Chameleons` are included in the  `input/` directory. \n\u003c/p\u003e\n### Options\n\nLearning of the embedding is handled by the `src/main.py` script which provides the following command line arguments.\n\n#### Input and output options\n\n```\n  --edge-path         STR   Input graph path.     Default is `input/chameleon_edges.csv`.\n  --embedding-path    STR   Embedding path.       Default is `output/chameleon_AW_embedding.csv`.\n  --attention-path    STR   Attention path.       Default is `output/chameleon_AW_attention.csv`.\n```\n\n#### Model options\n\n```\n  --dimensions           INT       Number of embeding dimensions.        Default is 128.\n  --epochs               INT       Number of training epochs.            Default is 200.\n  --window-size          INT       Skip-gram window size.                Default is 5.\n  --learning-rate        FLOAT     Learning rate value.                  Default is 0.01.\n  --beta                 FLOAT     Attention regularization parameter.   Default is 0.5.\n  --gamma                FLOAT     Embedding regularization parameter.   Default is 0.5.\n  --num-of-walks         INT       Number of walks per source node.      Default is 80.\n```\n\n### Examples\n\u003cp align=\"justify\"\u003e\nThe following commands learn a graph embedding and write the embedding to disk. The node representations are ordered by the ID.\n\u003c/p\u003e\n\u003cp align=\"justify\"\u003e\nCreating an Attention Walk embedding of the default dataset with the standard hyperparameter settings. Saving this embedding at the default path.\n\u003c/p\u003e\n```\npython src/main.py\n```\n\u003cp align=\"center\"\u003e\n\u003cimg style=\"float: center;\" src=\"attention_walk_run_example.jpg\"\u003e\n\u003c/p\u003e\n\nCreating an Attention Walk embedding of the default dataset with 256 dimensions.\n```\npython src/main.py --dimensions 256\n```\n\nCreating an Attention Walk embedding of the default dataset with a higher window size.\n\n```\npython src/main.py --window-size 20\n```\n\nCreating an embedding of another dataset the `Twitch Brasilians`. Saving the outputs under custom file names.\n\n```\npython src/main.py --edge-path input/ptbr_edges.csv --embedding-path output/ptbr_AW_embedding.csv --attention-path output/ptbr_AW_attention.csv\n```\n--------------------------------------------------------------------------------\n\n**License**\n\n- [GNU License](https://github.com/benedekrozemberczki/AttentionWalk/blob/master/LICENSE)\n\n--------------------------------------------------------------------------------\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbenedekrozemberczki%2Fattentionwalk","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbenedekrozemberczki%2Fattentionwalk","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbenedekrozemberczki%2Fattentionwalk/lists"}