{"id":13481133,"url":"https://github.com/edouardpineau/Variational-Recurrent-Neural-Networks-for-Graph-Classification","last_synced_at":"2025-03-27T11:31:53.837Z","repository":{"id":105015539,"uuid":"190383889","full_name":"edouardpineau/Variational-Recurrent-Neural-Networks-for-Graph-Classification","owner":"edouardpineau","description":"Github page for the paper \"Variational Recurrent Neural Networks for Graph Classification\" presented at the RLGM workshop of ICLR 2019","archived":false,"fork":false,"pushed_at":"2020-10-21T07:35:17.000Z","size":666,"stargazers_count":13,"open_issues_count":0,"forks_count":4,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-10-30T14:43:36.254Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/edouardpineau.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-06-05T11:40:51.000Z","updated_at":"2024-09-22T18:59:45.000Z","dependencies_parsed_at":null,"dependency_job_id":"ad58a0f5-091c-4fb3-93ca-a8e966c374db","html_url":"https://github.com/edouardpineau/Variational-Recurrent-Neural-Networks-for-Graph-Classification","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/edouardpineau%2FVariational-Recurrent-Neural-Networks-for-Graph-Classification","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/edouardpineau%2FVariational-Recurrent-Neural-Networks-for-Graph-Classification/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/edouardpineau%2FVariational-Recurrent-Neural-Networks-for-Graph-Classification/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/edouardpineau%2FVariational-Recurrent-Neural-Networks-for-Graph-Classification/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/edouardpineau","download_url":"https://codeload.github.com/edouardpineau/Variational-Recurrent-Neural-Networks-for-Graph-Classification/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245836252,"owners_count":20680339,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T17:00:49.001Z","updated_at":"2025-03-27T11:31:53.453Z","avatar_url":"https://github.com/edouardpineau.png","language":"Python","funding_links":[],"categories":["Deep Learning"],"sub_categories":[],"readme":"# Variational Recurrent Neural Networks for Graph Classification\n\nThis work was presented at the ICLR 2019 workshop session [Representation Learning on Graphs and Manifolds](https://rlgm.github.io/). \n\nOur paper can be found here: https://rlgm.github.io/papers/9.pdf\n\nThe notebook VRGC.ipynb requires the data that can be downloaded [here](https://ls11-www.cs.tu-dortmund.de/staff/morris/graphkerneldatasets).\n\n### Abstract\n\nWe address the problem of graph classification based only on structural information. Inspired by natural language processing techniques (NLP), our model sequentially embeds information to estimate class membership probabilities. Besides, we experiment with NLP-like variational regularization techniques, making the model predict the next node in the sequence as it reads it. We experimentally show that our model achieves state-of-the-art classification results on several standard molecular datasets. Finally, we perform a qualitative analysis and give some insights on whether the node prediction helps the model better classify graphs.\n\n### Multi-task learning\n\nMulti-task learning is a powerful leverage to learn rich representation in NLP [1]. We propose to use it for our problem.\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"https://github.com/edouardpineau/Variational-Recurrent-Neural-Networks-for-Graph-Classification/raw/master/images/archi_macro.png\" width=\"700\"\u003e\u003c/p\u003e\n\n#### Graph preprocessing\n\nWe use a BFS node ordering procedure to transform graph into sequence of nodes as in [2]. \n\n##### Breadth-first search with random root R for graph enumeration\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"https://github.com/edouardpineau/Variational-Recurrent-Neural-Networks-for-Graph-Classification/raw/master/images/BFS.png\" width=\"400\"\u003e\u003c/p\u003e\n\u003cp align=\"center\"\u003eFigure 2: Example of a BFS node ordering.\u003c/p\u003e\n\n##### Sequencial truncated node adjacency\n\nEach node is only related to its two closest neighbors in the order of the BFS to get a low dimensional sequence of nodes. \n\n\u003cp align=\"center\"\u003e\u003cimg src=\"https://github.com/edouardpineau/Variational-Recurrent-Neural-Networks-for-Graph-Classification/raw/master/images/SeqAdj.png\" width=\"500\"\u003e\u003c/p\u003e\n\u003cp align=\"center\"\u003eFigure 3: Example of a BFS node ordering.\u003c/p\u003e\n\n##### Complete graph-to-sequence embedding\n\nEach node embedding contains both current and previous node BFS-related adjacency information thanks to RNN memory structure.\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"https://github.com/edouardpineau/Variational-Recurrent-Neural-Networks-for-Graph-Classification/raw/master/images/archi_0.png\" width=\"600\"\u003e\u003c/p\u003e\n\u003cp align=\"center\"\u003eFigure 4: Example of a BFS node ordering.\u003c/p\u003e\n\n#### Recurrent neural network for sequence classification\n\nThe fully connected (FC) classifier is fed with sequence of the truncated BFS-ordered embedded node sequence.\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"https://github.com/edouardpineau/Variational-Recurrent-Neural-Networks-for-Graph-Classification/raw/master/images/archi_2.png\" width=\"600\"\u003e\u003c/p\u003e\n\u003cp align=\"center\"\u003eFigure 5: Recurrent classifier for for sequence classification.\u003c/p\u003e\n\n#### Variational autoregressive (VAR) node prediction\n\nA node prediction task is added to help the classifier. The task is performed by a variational autoencoder feed with the same sequence of embedded nodes than the recurrent classifier.\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"https://github.com/edouardpineau/Variational-Recurrent-Neural-Networks-for-Graph-Classification/raw/master/images/archi_1.png\" width=\"600\"\u003e\u003c/p\u003e\n\u003cp align=\"center\"\u003eFigure 6: Variational autoregressive node prediction.\u003c/p\u003e\n\n\n### Results\n\n- VRGC is not structurally invariant to node indexing, like in our previous [graph classification work](https://github.com/edouardpineau/A-simple-baseline-algorithm-for-graph-classification). However our model learns node indexing invariance from numerous training iterations on randomly-rooted BFS-ordered sequential graph embedding.\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"https://github.com/edouardpineau/Variational-Recurrent-Neural-Networks-for-Graph-Classification/raw/master/images/Projection.png\" width=\"400\"\u003e\u003c/p\u003e\n\u003cp align=\"center\"\u003eFigure 7: TSNE projection of the latent state preceding classification for five graphs from four distinct classes, each initiated with 20 different BFS. Colors and markers represent the respective classes of the graphs.\u003c/p\u003e\n\n\u0026nbsp;\n\n- VAR helps the model finding a more meaningful latent representation for classification while graph dataset becomes larger, with marginal extra computational cost with respect to RNNs.\n\n\u003cp align=\"center\"\u003e\u003cimg src=\"https://github.com/edouardpineau/Variational-Recurrent-Neural-Networks-for-Graph-Classification/raw/master/images/table.png\" width=\"500\"\u003e\u003c/p\u003e\n\u003cp align=\"center\"\u003eFigure 8: Classification results.\u003c/p\u003e\n\n### Citing\n\n    @article{pineau2019variational,\n              title={Variational recurrent neural networks for graph classification},\n              author={Pineau, Edouard and de Lara, Nathan},\n              journal={Representation Learning on Graphs and Manifolds Workshop (ICLR)},\n              year={2019}\n}\n\n\n### Datasets and references\n\nAll datasets can be found here: https://ls11-www.cs.tu-dortmund.de/staff/morris/graphkerneldatasets\n\n[1] V. Sanh, T. Wolf, and S. Ruder. A hierarchical multi-task approach for learning embeddings from semantic tasks. arXiv preprint arXiv:1811.06031, 2018.\n\n[2] GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models, Jiaxuan You, Rex Ying, Xiang Ren, William Hamilton, Jure Leskovec ; Proceedings of the 35th International Conference on Machine Learning, PMLR 80:5708-5717, 2018.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fedouardpineau%2FVariational-Recurrent-Neural-Networks-for-Graph-Classification","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fedouardpineau%2FVariational-Recurrent-Neural-Networks-for-Graph-Classification","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fedouardpineau%2FVariational-Recurrent-Neural-Networks-for-Graph-Classification/lists"}