{"id":27099499,"url":"https://github.com/puzzlef/louvain-communities-dynamic","last_synced_at":"2025-04-06T12:35:46.469Z","repository":{"id":249712982,"uuid":"538336155","full_name":"puzzlef/louvain-communities-dynamic","owner":"puzzlef","description":"Comparing static vs dynamic approaches of the Louvain algorithm for community detection.","archived":false,"fork":false,"pushed_at":"2024-07-22T18:15:00.000Z","size":96,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-07-22T23:07:34.533Z","etag":null,"topics":["aggregation","algorithm","community","detection","experiment","graph","iterative","local","louvain","moving"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/puzzlef.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-09-19T04:42:37.000Z","updated_at":"2024-07-22T23:07:43.349Z","dependencies_parsed_at":"2024-07-22T23:07:41.554Z","dependency_job_id":"ede5cfda-a866-4b92-b2c6-b46e7f8075d8","html_url":"https://github.com/puzzlef/louvain-communities-dynamic","commit_stats":null,"previous_names":["puzzlef/louvain-communities-dynamic"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/puzzlef%2Flouvain-communities-dynamic","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/puzzlef%2Flouvain-communities-dynamic/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/puzzlef%2Flouvain-communities-dynamic/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/puzzlef%2Flouvain-communities-dynamic/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/puzzlef","download_url":"https://codeload.github.com/puzzlef/louvain-communities-dynamic/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247485270,"owners_count":20946397,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aggregation","algorithm","community","detection","experiment","graph","iterative","local","louvain","moving"],"created_at":"2025-04-06T12:35:46.052Z","updated_at":"2025-04-06T12:35:46.446Z","avatar_url":"https://github.com/puzzlef.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"Comparing *static* vs *dynamic* approaches of the [Louvain algorithm] for\n[community detection].\n\n[Louvain] is an algorithm for **detecting communities in graphs**. *Community*\n*detection* helps us understand the *natural divisions in a network* in an\n**unsupervised manner**. It is used in **e-commerce** for *customer*\n*segmentation* and *advertising*, in **communication networks** for *multicast*\n*routing* and setting up of *mobile networks*, and in **healthcare** for\n*epidemic causality*, setting up *health programmes*, and *fraud detection* is\nhospitals. *Community detection* is an **NP-hard** problem, but heuristics exist\nto solve it (such as this). **Louvain algorithm** is an **agglomerative-hierarchical**\ncommunity detection method that **greedily optimizes** for **modularity**\n(**iteratively**).\n\n**Modularity** is a score that measures *relative density of edges inside* vs\n*outside* communities. Its value lies between `−0.5` (*non-modular clustering*)\nand `1.0` (*fully modular clustering*). Optimizing for modularity *theoretically*\nresults in the best possible grouping of nodes in a graph.\n\nGiven an *undirected weighted graph*, all vertices are first considered to be\n*their own communities*. In the **first phase**, each vertex greedily decides to\nmove to the community of one of its neighbors which gives greatest increase in\nmodularity. If moving to no neighbor's community leads to an increase in\nmodularity, the vertex chooses to stay with its own community. This is done\nsequentially for all the vertices. If the total change in modularity is more\nthan a certain threshold, this phase is repeated. Once this **local-moving**\n**phase** is complete, all vertices have formed their first hierarchy of\ncommunities. The **next phase** is called the **aggregation phase**, where all\nthe *vertices belonging to a community* are *collapsed* into a single\n**super-vertex**, such that edges between communities are represented as edges\nbetween respective super-vertices (edge weights are combined), and edges within\neach community are represented as self-loops in respective super-vertices\n(again, edge weights are combined). Together, the local-moving and the\naggregation phases constitute a **pass**. This super-vertex graph is then used\nas input for the next pass. This process continues until the increase in\nmodularity is below a certain threshold. As a result from each pass, we have a\n*hierarchy of community memberships* for each vertex as a **dendrogram**. We\ngenerally consider the *top-level hierarchy* as the *final result* of community\ndetection process.\n\n*Louvain* algorithm is a hierarchical algorithm, and thus has two different\ntolerance parameters: `tolerance` and `passTolerance`. **tolerance** defines the\nminimum amount of increase in modularity expected, until the local-moving phase\nof the algorithm is considered to have converged. We compare the increase in\nmodularity in each iteration of the local-moving phase to see if it is below\n`tolerance`. **passTolerance** defines the minimum amount of increase in\nmodularity expected, until the entire algorithm is considered to have converged.\nWe compare the increase in modularity across all iterations of the local-moving\nphase in the current pass to see if it is below `passTolerance`. `passTolerance`\nis normally set to `0` (we want to maximize our modularity gain), but the same\nthing does not apply for `tolerance`. Adjusting values of `tolerance` between\neach pass have been observed to impact the runtime of the algorithm, without\nsignificantly affecting the modularity of obtained communities. In this\nexperiment, we compare the performance of *three different types* of **dynamic**\n**Louvain** with respect to the *static* version.\n\n**Naive dynamic**:\n- We start with previous community membership of each vertex (instead of each vertex its own community).\n\n**Delta screening**:\n- All edge batches are undirected, and sorted by source vertex-id.\n- For edge additions across communities with source vertex `i` and highest modularity changing edge vertex `j*`,\n  `i`'s neighbors and `j*`'s community is marked as affected.\n- For edge deletions within the same community `i` and `j`,\n  `i`'s neighbors and `j`'s community is marked as affected.\n\n**Frontier-based**:\n- All source and destination vertices are marked as affected for insertions and deletions.\n- For edge additions across communities with source vertex `i` and destination vertex `j`,\n  `i` is marked as affected.\n- For edge deletions within the same community `i` and `j`,\n  `i` is marked as affected.\n- Vertices whose communities change in local-moving phase have their neighbors marked as affected.\n\nThe input data used for the experiments is available from the\n[SuiteSparse Matrix Collection]. These experiments are done with guidance\nfrom [Prof. Kishore Kothapalli] and [Prof. Dip Sankar Banerjee].\n\n\u003cbr\u003e\n\n\n### Comparing various Naive-dynamic approaches\n\nIn this experiment ([approaches-naive]), we compare the performance of *two*\n*different types* of **naive dynamic Louvain** with respect to the *static*\nversion. The **last** approach (`louvainSeqDynamicLast`) considers the community\nmembership of each vertex *after* the Louvain algorithm has *converged*\n(community membership from the \"last\" pass) and then performs the Louvain\nalgorithm upon the new (updated) graph. This is *similar* to naive dynamic\napproaches with other algorithms. On the other hand, the **first** approach\n(`louvainSeqDynamicFirst`) considers the community membership of each vertex\nright after the *first pass* of the Louvain algorithm (this is the first\ncommunity membership hierarchy) and then performs the Louvain algorithm upon the\nupdated graph. With this approach, we allow the affected vertices to choose\ntheir community membership from the first pass itself, which which to my\nintuition would lead to better communities.\n\nFirst, we compute the community membership of each vertex using the static\nLouvain algorithm (`louvainSeqLast`). We also run the static Louvain algorithm\nfor only one pass (`louvainSeqFirst`). We then generate *batches* of *insertions*\n*(+)* and *deletions (-)* of edges of sizes 500, 1000, 5000, ... 100000. For each\nbatch size, we generate *five* different batches for the purpose of *averaging*.\nEach batch of edges (insertion / deletion) is generated randomly such that the\nselection of each vertex (as endpoint) is *equally probable*. We choose the\nLouvain *parameters* as `resolution = 1.0`, `tolerance = 1e-2` (for local-moving\nphase) with *tolerance* decreasing after every pass by a factor of\n`toleranceDeclineFactor = 10`, and a `passTolerance = 0.0` (when passes stop).\nIn addition we limit the maximum number of iterations in a single local-moving\nphase with `maxIterations = 500`, and limit the maximum number of passes with\n`maxPasses = 500`. We run the Louvain algorithm until convergence (or until the\nmaximum limits are exceeded), and measure the **time** **taken** for the\n*computation* (performed 5 times for averaging), the **modularity score**, the\n**total number of iterations** (in the *local-moving* *phase*), and the number\nof **passes**. This is repeated for *seventeen* different graphs.\n\nFrom the results, we make make the following observations. The performance of\ndynamic approaches upon a batch of deletions appears to *increase* with *increasing*\nbatch size*. This makes sense since, as the graph keeps getting smaller, the\ncomputation would complete *sooner*. Next, the `first` naive dynamic approach is\nfound to be *significantly slower* (~0.3x speedup) than the `last` approach.\nHowever, the `first` approach is *still faster* than the static approach upto a\nbatch size of `50000`. On the other hand, the `last` approach is *faster* than the\nstatic approach for all batch sizes. A similar behavior is observed with the\ntotal number of iterations. The `first` approach seems to have a *slightly higher*\nmodularity with respect to the `last` approach. Since the modularity between the\ntwo dynamic approaches are almost the same, the **last** approach is clearly the\n**best choice**.\n\n[approaches-naive]: https://github.com/puzzlef/louvain-communities-dynamic/tree/approaches-naive\n\n\u003cbr\u003e\n\n\n### Comparision with Static approach\n\nFirst ([compare-static], [main]), we compute the community membership of each vertex\nusing the static Louvain algorithm. We then generate *batches* of *insertions*\n*(+)* and *deletions (-)* of edges of sizes 500, 1000, 5000, ... 100000. For\neach batch size, we generate *five* different batches for the purpose of\n*averaging*. Each batch of edges (insertion / deletion) is generated randomly\nsuch that the selection of each vertex (as endpoint) is *equally probable*. We\nchoose the Louvain *parameters* as `resolution = 1.0`, `tolerance = 1e-2` (for\nlocal-moving phase) with *tolerance* decreasing after every pass by a factor of\n`toleranceDeclineFactor = 10`, and a `passTolerance = 0.0` (when passes stop).\nIn addition we limit the maximum number of iterations in a single local-moving\nphase with `maxIterations = 500`, and limit the maximum number of passes with\n`maxPasses = 500`. We run the Louvain algorithm until convergence (or until the\nmaximum limits are exceeded), and measure the **time taken** for the\n*computation* (performed 5 times for averaging), the **modularity score**, the\n**total number of iterations** (in the *local-moving* *phase*), and the number\nof **passes**. This is repeated for *seventeen* different graphs.\n\nFrom the results, we make make the following observations. The frontier-based\ndynamic approach converges the fastest, which obtaining communities with only\nslightly lower modularity than other approaches. We also observe that\ndelta-screening based dynamic Louvain algorithm has the same performance as that\nof the naive dynamic approach. Therefore, **frontier-based dynamic Louvain**\nwould be the **best choice**. All outputs are saved in a [gist]. Some [charts]\nare also included below, generated from [sheets].\n\n[![](https://i.imgur.com/MVFks1a.png)][sheetp]\n[![](https://i.imgur.com/du5qe69.png)][sheetp]\n[![](https://i.imgur.com/AulWzoj.png)][sheetp]\n[![](https://i.imgur.com/bxpYvsF.png)][sheetp]\n\n[compare-static]: https://github.com/puzzlef/louvain-communities-dynamic/tree/compare-static\n[main]: https://github.com/puzzlef/louvain-communities-dynamic\n\n\u003cbr\u003e\n\n\u003cbr\u003e\n\u003cbr\u003e\n\n\n## References\n\n- [Fast unfolding of communities in large networks; Vincent D. Blondel et al. (2008)](https://arxiv.org/abs/0803.0476)\n- [Community Detection on the GPU; Md. Naim et al. (2017)](https://arxiv.org/abs/1305.2006)\n- [Scalable Static and Dynamic Community Detection Using Grappolo; Mahantesh Halappanavar et al. (2017)](https://ieeexplore.ieee.org/document/8091047)\n- [From Louvain to Leiden: guaranteeing well-connected communities; V.A. Traag et al. (2019)](https://www.nature.com/articles/s41598-019-41695-z)\n- [Community detection in networks: A multidisciplinary review; Muhammad Aqib Javed et al. (2018)](https://www.sciencedirect.com/science/article/abs/pii/S1084804518300560)\n- [Adaptive parallel Louvain community detection on a multicore platform; Mahmood Fazlali et al. (2017)](https://www.sciencedirect.com/science/article/abs/pii/S014193311630240X)\n- [A signal processing perspective to community detection in dynamic networks; Selin Aviyente (2021)](https://www.sciencedirect.com/science/article/abs/pii/S1051200421002311)\n- [Incremental Community Detection in Distributed Dynamic Graph; Tariq Abughofa et al. (2021)](https://ieeexplore.ieee.org/abstract/document/9564329)\n- [Community Detection in Dynamic Attributed Graphs; Gonzalo A. Bello et al. (2016)](https://link.springer.com/chapter/10.1007/978-3-319-49586-6_22)\n- [Dynamic Community Detection Using Nonnegative Matrix Factorization; Feng Gao et al. (2017)](https://ieeexplore.ieee.org/abstract/document/8327706)\n- [Improving fuzzy C-mean-based community detection in social networks using dynamic parallelism; Mahmoud Al-Ayyoub et al. (2019)](https://www.sciencedirect.com/science/article/abs/pii/S0045790617315641)\n- [Distributed Louvain Algorithm for Graph Community Detection; Sayan Ghosh et al. (2018)](https://ieeexplore.ieee.org/abstract/document/8425242)\n- [Evaluating community detection algorithms for progressively evolving graphs; Remy Cazabet et al. (2020)](https://academic.oup.com/comnet/article-abstract/8/6/cnaa027/6161494?login=false)\n- [Scalable distributed Louvain algorithm for community detection in large graphs; Naw Safrin Sattar et al. (2022)](https://link.springer.com/article/10.1007/s11227-021-04224-2)\n- [Community Detection in Evolving Networks; Tejas Puranik et al. (2017)](https://ieeexplore.ieee.org/abstract/document/9069125)\n- [Incremental local community identification in dynamic social networks; Mansoureh Takaffoli et al. (2013)](https://ieeexplore.ieee.org/abstract/document/6785692)\n- [Dynamic community detection based on the Matthew effect; Zejun Sun et al. (2022)](https://www.sciencedirect.com/science/article/abs/pii/S0378437122002564)\n- [Community detection in social networks; Punam Bedi et al. (2016)](https://wires.onlinelibrary.wiley.com/doi/abs/10.1002/widm.1178)\n- [Long range community detection; Thomas Aynaud et al. (2010)](https://inria.hal.science/inria-00531750/)\n- [Dynamic Community Detection Algorithm Based on Incremental Identification; Xiaoming Li et al. (2015)](https://ieeexplore.ieee.org/abstract/document/7395763)\n- [An Improved Louvain Algorithm for Community Detection; Jicun Zhang et al. (2021)](https://onlinelibrary.wiley.com/doi/10.1155/2021/1485592)\n- [DyPerm: Maximizing Permanence for Dynamic Community Detection; Prerna Agarwal et al. (2018)](https://link.springer.com/chapter/10.1007/978-3-319-93034-3_35)\n- [A fast and efficient incremental approach toward dynamic community detection; Neda Zarayeneh et al. (2019)](https://dl.acm.org/doi/abs/10.1145/3341161.3342877)\n- [Scalable Community Detection with the Louvain Algorithm; Xinyu Que et al. (2015)](https://ieeexplore.ieee.org/abstract/document/7161493)\n- [DynaMo: Dynamic Community Detection by Incrementally Maximizing Modularity; Di Zhuang et al. (2019)](https://ieeexplore.ieee.org/abstract/document/8890861)\n- [C-Blondel: An Efficient Louvain-Based Dynamic Community Detection Algorithm; Mahsa Seifikar et al. (2020)](https://ieeexplore.ieee.org/abstract/document/8982190)\n- [Scalable static and dynamic community detection using Grappolo; Mahantesh Halappanavar et al. (2017)](https://ieeexplore.ieee.org/abstract/document/8091047)\n- [Dynamic community detection in evolving networks using locality modularity optimization; Mário Cordeiro et al. (2016)](https://link.springer.com/article/10.1007/s13278-016-0325-1)\n- [A Dynamic Modularity Based Community Detection Algorithm for Large-scale Networks: DSLM; Riza Aktunc et al. (2015)](https://dl.acm.org/doi/abs/10.1145/2808797.2808822)\n- [An Analysis of the Dynamic Community Detection Algorithms in Complex Networks; Dhananjay Kumar Singh et al. (2020)](https://ieeexplore.ieee.org/abstract/document/9067224)\n- [Dynamic Community Detection Decouples Multiple Time Scale Behavior of Complex Chemical Systems; Neda Zarayeneh et al. (2022)](https://pubs.acs.org/doi/full/10.1021/acs.jctc.2c00454)\n- [Targeted revision: A learning-based approach for incremental community detection in dynamic networks; Jiaxing Shang et al. (2016)](https://www.sciencedirect.com/science/article/abs/pii/S0378437115008080)\n- [DynComm R Package -- Dynamic Community Detection for Evolving Networks; Rui Portocarrero Sarmento et al. (2019)](https://arxiv.org/abs/1905.01498)\n- [CS224W: Machine Learning with Graphs | Louvain Algorithm; Jure Leskovec (2021)](https://www.youtube.com/watch?v=0zuiLBOIcsw)\n- [The University of Florida Sparse Matrix Collection; Timothy A. Davis et al. (2011)](https://doi.org/10.1145/2049662.2049663)\n\n\u003cbr\u003e\n\u003cbr\u003e\n\n\n[![](https://i.imgur.com/UGB0g2L.jpg)](https://www.youtube.com/watch?v=pIF3wOet-zw)\u003cbr\u003e\n[![ORG](https://img.shields.io/badge/org-puzzlef-green?logo=Org)](https://puzzlef.github.io)\n[![DOI](https://zenodo.org/badge/538336155.svg)](https://zenodo.org/badge/latestdoi/538336155)\n\n\n[Prof. Dip Sankar Banerjee]: https://sites.google.com/site/dipsankarban/\n[Prof. Kishore Kothapalli]: https://faculty.iiit.ac.in/~kkishore/\n[SuiteSparse Matrix Collection]: https://sparse.tamu.edu\n[Louvain algorithm]: https://en.wikipedia.org/wiki/Louvain_method\n[community detection]: https://en.wikipedia.org/wiki/Community_search\n[Louvain]: https://en.wikipedia.org/wiki/Louvain_method\n[gist]: https://gist.github.com/wolfram77/de2c1e1c8f6efb7f4053a122b688c7a7\n[charts]: https://imgur.com/a/xcoVmDw\n[sheets]: https://docs.google.com/spreadsheets/d/1U8cdi0Y9i-Pl0SkUdSsuM87Khw0Jn6SCM7u6qlVp--Y/edit?usp=sharing\n[sheetp]: https://docs.google.com/spreadsheets/d/e/2PACX-1vQjZGdwSy2Cd5mojCQgvvl0P5eaZBjJwhIBqcX-RN5MfeFVl9V2o9G4XCjR2_qaH_mpeBSm7eMIhSeQ/pubhtml\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpuzzlef%2Flouvain-communities-dynamic","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpuzzlef%2Flouvain-communities-dynamic","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpuzzlef%2Flouvain-communities-dynamic/lists"}