{"id":32639866,"url":"https://github.com/filippomb/total-variation-graph-neural-networks","last_synced_at":"2025-10-31T02:16:29.254Z","repository":{"id":160527122,"uuid":"563381561","full_name":"FilippoMB/Total-variation-graph-neural-networks","owner":"FilippoMB","description":"Pytorch and Tensorflow implementation of TVGNN, presented at ICML 2023.","archived":false,"fork":false,"pushed_at":"2025-03-15T12:10:32.000Z","size":1566,"stargazers_count":20,"open_issues_count":0,"forks_count":3,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-15T13:20:43.457Z","etag":null,"topics":["clustering","community-detection","graph-clustering","graph-neural-network","graph-pooling","spectral-clustering","total-variation-loss","unsupervised-learning"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2211.06218","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/FilippoMB.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-11-08T13:52:42.000Z","updated_at":"2025-03-15T12:10:36.000Z","dependencies_parsed_at":"2024-02-08T12:30:08.699Z","dependency_job_id":"6f7ae98f-7d9e-4b1b-a7f5-9ce9bc608959","html_url":"https://github.com/FilippoMB/Total-variation-graph-neural-networks","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/FilippoMB/Total-variation-graph-neural-networks","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FilippoMB%2FTotal-variation-graph-neural-networks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FilippoMB%2FTotal-variation-graph-neural-networks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FilippoMB%2FTotal-variation-graph-neural-networks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FilippoMB%2FTotal-variation-graph-neural-networks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/FilippoMB","download_url":"https://codeload.github.com/FilippoMB/Total-variation-graph-neural-networks/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/FilippoMB%2FTotal-variation-graph-neural-networks/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":281914594,"owners_count":26583095,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-31T02:00:07.401Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clustering","community-detection","graph-clustering","graph-neural-network","graph-pooling","spectral-clustering","total-variation-loss","unsupervised-learning"],"created_at":"2025-10-31T02:16:00.077Z","updated_at":"2025-10-31T02:16:29.246Z","avatar_url":"https://github.com/FilippoMB.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![ICML](https://img.shields.io/badge/ICML-2023-blue)](https://icml.cc/virtual/2023/poster/24747)\n[![arXiv](https://img.shields.io/badge/arXiv-2211.06218-b31b1b.svg)](https://arxiv.org/abs/2211.06218)\n[![Poster](https://img.shields.io/badge/%E2%87%A9-Poster-%23228B22.svg)](https://github.com/FilippoMB/Total-variation-graph-neural-networks/blob/main/tvgnn_poster.pdf)\n[![Video](https://img.shields.io/badge/Presentation-%23FF0000.svg?logo=YouTube\u0026logoColor=white)](https://youtu.be/Dyb1YJOez8w)\n\nTensorflow and Pytorch implementation of the Total Variation Graph Neural Network (TVGNN) as presented in the [original paper](https://arxiv.org/abs/2211.06218).\n\nThe TVGNN model can be used to **cluster** the vertices of an annotated graph, by accounting both for the graph topology and the vertex features. Compared to other GNNs for clustering, TVGNN creates *sharp* cluster assignments that better approximate the optimal (in the minimum cut sense) partition.\n\n\u003cimg align=\"center\" width=\"273\" height=\"143\" src=\"img/sharp.png\" alt=\"smooth and sharp clustering assignments\"\u003e\n\nThe TVGNN model can also be used to implement [graph pooling](https://gnn-pooling.notion.site/) in a deep GNN architecture for tasks such as graph classification.\n\n# Downstream tasks\nTVGNN can be used to perform vertex clustering and graph classification. Other tasks such as graph regression can also be done with the TVGNN model.\n\n### Vertex clustering\nThis is an unsupervised task, where the goal is to generate a partition of the vertices based on the similarity of their vertex features and the graph topology. The GNN model is trained only by minimizing the unsupervised loss $\\mathcal{L}$.\n\n\u003cimg align=\"center\" width=\"150\" height=\"105\" src=\"img/clustering.png\" alt=\"clustering architecture\"\u003e\n\n### Graph classification\nThis is a supervised with goal of predicting the class of each graph. The GNN rchitectures for graph classification alternates GTVConv layers with a graph pooling layer, which gradually distill the global label information from the vertex representations. The GNN is trained by minimizing the unsupervised loss $\\mathcal{L}$ for each pooling layer and a supervised cross-entropy loss $\\mathcal{L}_\\text{cross-entr}$ between the true and predicted class label.\n\n\u003cimg align=\"center\" width=\"497\" height=\"130\" src=\"img/classification.png\" alt=\"classification architecture\"\u003e\n\n# 💻 Implementation\n\n\u003cimg align=\"left\" width=\"30\" height=\"30\" src=\"https://upload.wikimedia.org/wikipedia/commons/2/2d/Tensorflow_logo.svg\" alt=\"Tensorflow icon\"\u003e\n\n### Tensorflow\nThis implementation is based on the [Spektral](https://graphneural.network/) library and follows the [Select-Reduce-Connect](https://graphneural.network/layers/pooling/#srcpool) API.\nTo execute the code, first install the conda environment from [tf_environment.yml](tensorflow/tf_environment.yml)\n\n    conda env create -f tf_environment.yml\n\nThe ``tensorflow/`` folder includes:\n\n- The implementation of the [GTVConv](/tensorflow/GTVConv.py) layer\n- The implementation of the [AsymCheegerCutPool](/tensorflow/AsymCheegerCutPool.py) layer\n- An example script to perform the [clustering](/tensorflow/clustering.py) task\n- An example script to perform the  [classification](/tensorflow/classification.py) task\n\n\u003cimg align=\"left\" width=\"30\" height=\"30\" src=\"https://upload.wikimedia.org/wikipedia/commons/1/10/PyTorch_logo_icon.svg\" alt=\"Pytorch icon\"\u003e\n\n### Pytorch\nThis implementation is based on the [Pytorch Geometric](https://pytorch-geometric.readthedocs.io/) library. To execute the code, install the conda environment from [pytorch_environment.yml](pytorch/pytorch_environment.yml)\n\n    conda env create -f pytorch_environment.yml\n\nThe ``pytorch/`` folder includes:\n\n- The implementation of the [GTVConv](/pytorch/GTVConv.py) layer\n- The implementation of the [AsymCheegerCutPool](/pytorch/AsymCheegerCutPool.py) layer\n- An example script to perform the [clustering](/pytorch/clustering.py) task\n- An example script to perform the  [classification](/pytorch/classification.py) task\n\n\u003cimg align=\"left\" width=\"28\" height=\"30\" src=\"https://github.com/danielegrattarola/spektral/blob/master/docs/img/ghost_dark.svg\" alt=\"Tensorflow icon\"\u003e\n\n### Spektral\n\nTVGNN is available on Spektral:\n\n- [GTVConv](https://graphneural.network/layers/convolution/#gtvconv) layer,\n- [AsymCheegerCutPool](https://graphneural.network/layers/pooling/#asymcheegercutpool) layer,\n- [Example script](https://github.com/danielegrattarola/spektral/blob/master/examples/other/node_clustering_tvgnn.py) to perform node clustering with TVGNN.\n\n# 📚 Citation\nIf you use TVGNN in your research, please consider citing our work as\n\n````bibtex\n@inproceedings{hansen2023total,\n  title={Total variation graph neural networks},\n  author={Hansen, Jonas Berg and Bianchi, Filippo Maria},\n  booktitle={International Conference on Machine Learning},\n  pages={12445--12468},\n  year={2023},\n  organization={PMLR}\n}\n````\n\n# ⚙️ Technical details \nTVGNN consists of the GTVConv layer and the AsymmetricCheegerCut layer.\n\n### GTVConv\nThe GTVConv layer is a *message-passing* layer that minimizes the $L_1$-norm of the difference between features of adjacent nodes. The $l$-th GTVConv layer updates the node features as\n\n$$\\mathbf{X}^{(l+1)}  = \\sigma\\left[ \\left( \\mathbf{I} - 2\\delta \\mathbf{L}_\\Gamma^{(l)}  \\right) \\mathbf{X}^{(l)}\\mathbf{\\Theta}  \\right] $$ \n\nwhere $\\sigma$ is a non-lineary, $\\mathbf{\\Theta}$ are the trainable weights of the layer, and $\\delta$ is an hyperparameter. $\\mathbf{L}^{(l)}_ \\Gamma$ is a Laplacian defined as $\\mathbf{L}^{(l)}_ \\Gamma$ = $\\mathbf{D}^{(l)}_ \\Gamma - \\mathbf{\\Gamma}^{(l)}$, where $\\mathbf{D}_\\Gamma = \\text{diag}(\\mathbf{\\Gamma} \\boldsymbol{1})$ and \n\n$$ [\\mathbf{\\Gamma}]^{(l)}_ {ij} = \\frac{a_ {ij}}{\\texttt{max}\\{ \\lVert \\boldsymbol{x}_i^{(l)} - \\boldsymbol{x}_j^{(l)}  \\rVert_1, \\epsilon \\}}$$\n\nwhere $a_{ij}$ is the $ij$-th entry of the adjacency matrix, $\\boldsymbol{x}_i^{(l)}$ is the feature of vertex $i$ at layer $l$ and $\\epsilon$ is a small constant that avoids zero-division.\n\n### AsymCheegerCut\nThe AsymCheegerCut is a *graph pooling* layer that internally contains an $\\texttt{MLP}$ parametrized by $\\mathbf{\\Theta}_\\text{MLP}$ and that computes:\n- a cluster assignment matrix $\\mathbf{S} = \\texttt{Softmax}(\\texttt{MLP}(\\mathbf{X}; \\mathbf{\\Theta}_\\text{MLP})) \\in \\mathbb{R}^{N\\times K}$, which maps the $N$ vertices in $K$ clusters,\n- an unsupervised loss $\\mathcal{L} = \\alpha_1 \\mathcal{L}_ \\text{GTV} + \\alpha_2 \\mathcal{L}_ \\text{AN}$, where $\\alpha_1$ and $\\alpha_2$ are two hyperparameters,\n- the adjacency matrix and the vertex features of a coarsened graph\n\n$$\\mathbf{A}^\\text{pool} = \\mathbf{S}^T \\tilde{\\mathbf{A}} \\mathbf{S} \\in\\mathbb{R}^{K\\times K}; \\\\ \\mathbf{X}^\\text{pool}=\\mathbf{S}^T\\mathbf{X} \\in\\mathbb{R}^{K\\times F}.\n$$\n\nThe term $\\mathcal{L}_ \\text{GTV}$ in the loss minimizes the graph total variation of the cluster assignments $\\mathbf{S}$ and is defined as\n\n$$\\mathcal{L}_ \\text{GTV} = \\frac{\\mathcal{L}_ \\text{GTV}^*}{2E} \\in [0, 1],$$\n\nwhere $\\mathcal{L}_ \\text{GTV}^*$ = $\\displaystyle\\sum_{k=1}^K\\sum_{i=1}^N \\sum_{j=i}^N a_{i,j} |s_{i,k} - s_{j,k}|$, $s_{i,k}$ is the assignment of vertex $i$ to cluster $k$ and $E$ is the number of edges.\n\nThe term $\\mathcal{L}_\\text{AN}$ encourages the partition to be balanced and is defined as\n\n$$\\mathcal{L}_ {\\text{AN}} = \\frac{\\beta - \\mathcal{L}^*_ \\text{AN}}{\\beta} \\in [0, 1],$$\n\nwhere $\\mathcal{L}_ \\text{AN}^* = \\displaystyle\\sum^K_{k=1} ||\\boldsymbol{s}_ {:,k}$ - $\\text{quant}_ \\rho (\\boldsymbol{s}_ {:,k})||_ {1, \\rho}$.\nWhen $\\rho = K-1$, then $\\beta = N\\rho$.\nWhen $\\rho$ takes different values, then $\\beta = N\\rho\\min(1, K/(\\rho+1))$. \n$\\text{quant}_ \\rho(\\boldsymbol{s}_ k)$ denotes the $\\rho$-quantile of $\\boldsymbol{s}_ k$ and $||\\cdot||_ {1,\\rho}$ denotes an asymmetric $\\ell_1$ norm, which for a vector $\\boldsymbol{x} \\in \\mathbb{R}^{N\\times 1}$ is $||\\boldsymbol{x}||_ {1,\\rho}$ = $\\displaystyle\\sum^N_{i=1} |x_{i}|_ \\rho$, where $|x_i|_ \\rho = \\rho x_i$ if $x_i\\geq 0$ and $|x_i|_ \\rho = -x_i$ if  $x_i \u003c 0$. \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffilippomb%2Ftotal-variation-graph-neural-networks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffilippomb%2Ftotal-variation-graph-neural-networks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffilippomb%2Ftotal-variation-graph-neural-networks/lists"}