{"id":26404888,"url":"https://github.com/ebb-earl-co/libraries_io_cran","last_synced_at":"2026-04-10T20:46:32.836Z","repository":{"id":52699716,"uuid":"214033839","full_name":"ebb-earl-co/libraries_io_CRAN","owner":"ebb-earl-co","description":"Analyze contributors to CRAN using Libraries.io data ","archived":false,"fork":false,"pushed_at":"2024-01-10T04:38:15.000Z","size":512,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-14T16:03:33.808Z","etag":null,"topics":["cran","graph-algorithms","librariesio","neo4j","r"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ebb-earl-co.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-10-09T22:08:41.000Z","updated_at":"2024-02-06T15:33:28.000Z","dependencies_parsed_at":"2023-01-24T20:45:07.540Z","dependency_job_id":null,"html_url":"https://github.com/ebb-earl-co/libraries_io_CRAN","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ebb-earl-co%2Flibraries_io_CRAN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ebb-earl-co%2Flibraries_io_CRAN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ebb-earl-co%2Flibraries_io_CRAN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ebb-earl-co%2Flibraries_io_CRAN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ebb-earl-co","download_url":"https://codeload.github.com/ebb-earl-co/libraries_io_CRAN/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244074471,"owners_count":20393967,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cran","graph-algorithms","librariesio","neo4j","r"],"created_at":"2025-03-17T16:50:10.131Z","updated_at":"2026-04-10T20:46:27.813Z","avatar_url":"https://github.com/ebb-earl-co.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Finding the Most Influential CRAN Contributor using Neo4j and Libraries.io Open Data\n__For the first part of this project, to do with PyPi, see [here](https://github.com/ebb-earl-co/libraries_io)__\n\nAs a graduate student in statistics, I used `R` a lot. In fact, entire\nsemester-long courses were dedicated to learning how to harness some of\n`R`'s single-purpose (read: esoteric) packages for statistical modeling.\n\nBut when it came time for my capstone project, the data manipulation was\ndaunting... _until_ I discovered [`dyplr`](https://dplyr.tidyverse.org/).\nMoreover, I was completely taken by the paradigm outlined by `dyplr`'s author,\nHadley Wickham, in the\n[Split-Apply-Combine](https://vita.had.co.nz/papers/plyr.pdf) paper that he\nwrote, introducing what would become the guiding principle of the Tidyverse.\n\nSince then, the Tidyverse has exploded in popularity, becoming the\n[de facto standard](https://www.r-bloggers.com/why-learn-the-tidyverse/) for data\nmanipulation in `R`, and Hadley Wickham's veneration among `R` users has only\nincreased— and for good reason: now Python and `R` are the two most-used languages\nin data science.\n\nSo, the motivation for this project is akin to that of the aforementioned PyPi\ncontributors investigation: is Hadley Wickham the most influential `R` contributor?\nTo answer this question, we will analyze the `R` packages uploaded to\n[CRAN](https://cran.r-project.org); specifically:\n  * The `R` packages themselves\n  * What packages depend on what other packages\n  * Who contributes to what packages\n\nUsing these items, we will use the\n[degree centrality algorithm](https://en.wikipedia.org/wiki/Centrality#Degree_centrality)\nfrom graph theory to find the most influential node in the graph of `R` packages,\ndependencies, and contributors.\n## Summary of Results\nAfter constructing the graph (including imputing more than 2/3 of the `R` packages\nfrom Libraries.io Open Data dataset) and\n[analyzing the degree centrality](https://neo4j.com/docs/graph-algorithms/current/algorithms/degree-centrality/),\nHadley Wickham is indeed the most influential `R` contributor according to the\ndata from Libraries.io and CRAN. Below are the 10 `Contributor`s with the highest\ndegree centrality scores for this graph:\n\n|Contributor|GitHub login|Degree Centrality Score|\n|---|---|---|\n|Hadley Wickham|hadley|244 751|\n|Jim Hester|jimhester|170 123|\n|Kiril Müller|krlmlr|159 577|\n|Jennifer (Jenny) Bryan|jennybc|121 543|\n|Mara Averick|batpigandme|121 253|\n|Gábor Csárdi|gaborcsardi|101 351|\n|Hiroaki Yutani|yutannihilation|100 625|\n|Christophe Dervieux|cderv|98 078|\n|Jeroen Ooms|jeroen|82 055|\n|Craig Citro|craigcitro|71 207|\n\nFor insight into how this result was arrived at, read on.\n## The Approach\n### Libraries.io Open Data\n[CRAN](https://cran.r-project.org) is the repository for `R` packages that developers\nknow and love. Analogously to CRAN, other programming languages have their respective package\nmanagers, such as PyPi for Python. As a natural exercise in abstraction,\n[Libraries.io](https://libraries.io) is a meta-repository for\npackage managers. From [their website](https://libraries.io/data):\n\n\u003e Libraries.io gathers data from **36** package managers and **3** source code repositories.\nWe track over **2.7m** unique open source packages, **33m** repositories and **235m**\ninterdependencies between [sic] them. This gives Libraries.io a unique understanding of\nopen source software. An understanding that we want to share with **you**.\n\n#### Using Open Data Snapshot to Save API Calls\nLibraries.io has an easy-to-use [API](https://libraries.io/api), but\ngiven that CRAN has 15,000+ packages in the Open Data dataset,\nthe number of API calls to various endpoints to collate\nthe necessary data is not appealing (also, Libraries.io rate limits to 60 requests\nper minute). Fortunately, [Jeremy Katz on Zenodo](https://zenodo.org/record/2536573)\nmaintains snapshots of the Libraries.io Open Data source. The most recent\nversion is a snapshot from 22 December 2018, and contains the following CSV files:\n  1. Projects (3 333 927 rows)\n  2. Versions (16 147 579 rows)\n  3. Tags (52 506 651 rows)\n  4. Dependencies (105 811 885 rows)\n  5. Repositories (34 061 561 rows)\n  6. Repository dependencies (279 861 607 rows)\n  7. Projects with Related Repository Fields (3 343 749 rows)\n\nMore information about these CSVs is in the `README` file included in the Open\nData tar.gz, copied [here](https://github.com/ebb-earl-co/libraries_io_CRAN/blob/master/data/README).\nThere is a substantial reduction in the data when subsetting these CSVs just\nto the data pertaining to CRAN; find the code used to subset them and the\nsize comparisons [here](https://github.com/ebb-earl-co/libraries_io_CRAN/blob/master/data/cran_subsetting.md).\n\n**WARNING**: The tar.gz file that contains these data is 13 GB itself, and\nonce downloaded takes quite a while to un`tar`; once uncompressed, the data\ntake up 64 GB on disk!\n\n![untar time](images/untar_tar_gz_file_time.png)\n\n### Graph Databases, Starring Neo4j\nBecause of the interconnected nature of software packages (dependencies,\nversions, contributors, etc.), finding the most influential \"item\" in that web\nof data make [graph databases](https://db-engines.com/en/ranking/graph+dbms) and\n[graph theory](https://medium.freecodecamp.org/i-dont-understand-graph-theory-1c96572a1401)\nthe ideal tools for this type of analysis. [Neo4j](https://neo4j.com/product/)\nis the most popular graph database according to [DB engines](https://neo4j.com/product/),\nand is the one that we will use for the analysis. Part of the reason for its popularity\nis that its query language, [Cypher](https://neo4j.com/developer/cypher-query-language/),\nis expressive and simple:\n\n![example graph](images/example_graph.png)\n\nTerminology that will be useful going forward:\n  - `Jane Doe` and `John Smith` are __nodes__ (equivalently: __vertexes__)\n  - The above two nodes have __label__ `Person`, with __property__ `name`\n  - The line that connects the nodes is an __relationship__ (equivalently: __edge__)\n  - The above relationship is of __type__ `KNOWS`\n  - `KNOWS`, and all Neo4j relationships, are __directed__; i.e. `Jane Doe`\n  knows `John Smith`, but not the converse\n\nOn MacOS, the easiest way to use Neo4j is via the Neo4j Desktop app, available\nas the [`neo4j` cask on Homebrew](https://github.com/Homebrew/homebrew-cask/blob/master/Casks/neo4j.rb).\nNeo4j Desktop is a great IDE for Neo4j, allowing simple installation of different\nversions of Neo4j as well as plugins that are optional\n(e.g. [`APOC`](https://neo4j.com/docs/labs/apoc/current/)) but\nare really the best way to interact with the graph database. Moreover, the\nscreenshot above is taken from the Neo4j Browser, a nice interactive\ndatabase interface as well as query result visualization tool.\n#### Neo4j Configuration\nBefore we dive into the data model and how the data are loaded, Neo4j's\ndefault configuration isn't going to cut it for the packages and approach\nthat we are going to use, so the customized configuration file can be found\n[here](https://github.com/ebb-earl-co/libraries_io_CRAN/blob/master/data/neo4j.conf),\ncorresponding to Neo4j version 3.5.7.\n### Making a Graph of Libraries.io Open Data\n[Importing from CSV](https://neo4j.com/docs/cypher-manual/3.5/clauses/load-csv/)\nis the most common way to populate a Neo4j graph, and is how we will\nproceed given that the Open Data snapshot un`tar`s into CSV files. However,\nfirst a data model is necessary— what the entities that will be\nrepresented as labeled nodes with properties and the relationships\namong them are going to be. Moreover, some settings of Neo4j\nwill have to be customized for proper and timely import from CSV.\n#### Data Model\nBasically, when translating a data paradigm into graph data form, the nouns\nbecome nodes and how the nouns interact (the verbs) become the relationships.\nIn the case of the Libraries.io data, the following is the data model:\n\n![data model](images/graph.png)\n\nSo, a `Platform` `HOSTS` a `Project`, which `IS_WRITTEN_IN` a `Language`,\nand `HAS_VERSION` `Version`. Moreover, a `Project` `DEPENDS_ON` other\n`Project`s, and `Contributor`s `CONTRIBUTE_TO` `Project`s. With respect to\n`Version`s, the diagram communicates a limitation of the Libraries.io\nOpen Data: that `Project` nodes are linked in the dependencies CSV to other\n`Project` nodes, despite the fact that different versions of a project\ndepend on varying versions of other projects. Take, for example, this row\nfrom the [dependencies CSV](https://github.com/ebb-earl-co/libraries_io_CRAN/blob/master/data/cran_subsetting.md#dependencies):\n\n|ID|Project\\_Name|Project\\_ID|Version\\_Number|Version\\_ID|Dependency\\_Name|Dependency\\_Kind|Optional\\_Dependency|Dependency\\_Requirements|Dependency\\_Project\\_ID|\n|---|---|---|---|---|---|---|---|---|---|\n29033435|archivist|687281|1.0|7326353|RCurl|imports|false|\\*|688429|\n\nI.e.; `Version` 1.0 of `Project` `archivist` depends on `Project` `RCurl`.\nThere is no demarcation of __which__ version of `RCurl` it is that\nversion 1.0 of `archivist` depends on, other than `*` which forces the\nmodeling decision of `Project`s depending on other `Project`s, not `Version`s.\n#### Contributors, the Missing Data\nIt is impossible to answer the question of what contributor to CRAN is\nmost influential without, obviously, data on contributors. However, the\nOpen Data dataset lacks this information. In order to connect the Open\nData dataset with contributors data will require calls to the\n[Libraries.io API](https://libraries.io/api). As mentioned above, there\nis a rate limit of 60 requests per minute. If there are\n\n```bash\n$ mlr --icsv --opprint filter '$Platform == \"CRAN\"' then uniq -n -g \"ID\" projects-1.4.0-2018-12-22.csv\n 14455\n```\nPython-language Pypi packages, each of which sends one request to the\n[Contributors endpoint](https://libraries.io/api#project-contributors)\nof the Libraries.io API, at \"maximum velocity\", it will require\n\n![packages time to request](images/cran_packages_time_to_request.png)\n\nto get contributor data for each project.\n\nFollowing the example of\n[this blog](https://tbgraph.wordpress.com/2018/06/28/finding-alternative-routes-in-california-road-network-with-neo4j/),\nit is possible to use the aforementioned APOC utilities for Neo4j to\n[load data from web APIs](https://neo4j.com/docs/labs/apoc/current/import/web-apis/),\nbut I found it to be unwieldy and difficult to monitor. So, I used\nPython's `requests` and `SQLite` packages to send requests to the\nendpoint and store the responses in a long-running Bash process\n(code for this [here](https://github.com/ebb-earl-co/libraries_io_CRAN/blob/master/python/request_libraries_io_load_sqlite.py)).\n#### Database Constraints\nAnalogously to the unique constraint in a relational database, Neo4j has a\n[uniqueness constraint](https://neo4j.com/docs/cypher-manual/3.5/schema/constraints/#query-constraint-unique-nodes)\nwhich is very useful in constraining the number of nodes created. Basically,\nit isn't useful, and hurts performance, to have two different nodes representing the\nplatform Pypi (or the language Python, or the project `pipenv`, ...) because\nit is a unique entity. Moreover, uniqueness constraints enable\n[more performant queries](https://neo4j.com/docs/cypher-manual/3.5/clauses/merge/#query-merge-using-unique-constraints).\nThe following\n[Cypher commands](https://github.com/ebb-earl-co/libraries_io/blob/master/cypher/schema.cypher)\nadd uniqueness constraints on the properties of the nodes that should be unique\nin this data paradigm:\n```cypher\nCREATE CONSTRAINT on (platform:Platform) ASSERT platform.name IS UNIQUE;\nCREATE CONSTRAINT ON (project:Project) ASSERT project.name IS UNIQUE;\nCREATE CONSTRAINT ON (project:Project) ASSERT project.ID IS UNIQUE;\nCREATE CONSTRAINT ON (version:Version) ASSERT version.ID IS UNIQUE;\nCREATE CONSTRAINT ON (language:Language) ASSERT language.name IS UNIQUE;\nCREATE CONSTRAINT ON (contributor:Contributor) ASSERT contributor.uuid IS UNIQUE;\nCREATE INDEX ON :Contributor(name);\n```\nAll of the `ID` properties come from the first column of the CSVs and are\nostensibly primary key values. The `name` property of `Project` nodes is\nalso constrained to be unique so that queries seeking to match nodes on\nthe property name— the way that we think of them— are performant as well.\n\nN.b. if the graph from the first part of this analysis, concerning PyPi\npackages, is already populated, it will be necessary to drop the uniqueness\nconstraint on the `Project` names to avoid collisions. This is acceptable,\nas there will still be distinct `ID` values for the projects, but as `Project`\n`name` is the natural property to use for querying, a Neo4j\n[index](https://neo4j.com/docs/cypher-manual/current/schema/index/#schema-index-create-a-single-property-index)\nwill do the trick:\n```cypher\nDROP CONSTRAINT ON (p:Project) ASSERT p.name IS UNIQUE;\nCREATE INDEX ON :Project(name):\n```\n## Populating the Graph\nWith the constraints, plugins, and configuration of Neo4j in place,\nthe Libaries.io Open Data dataset can be loaded. Loading CSVs to Neo4j\ncan be done with the default\n[`LOAD CSV` command](https://neo4j.com/docs/cypher-manual/current/clauses/load-csv/),\nbut in the APOC plugin there is an improved version,\n[`apoc.load.csv`](https://neo4j.com/docs/labs/apoc/current/import/load-csv/#load-csv),\nwhich iterates over the CSV rows as map objects instead of arrays;\nwhen coupled with\n[periodic execution](https://neo4j.com/docs/labs/apoc/current/import/load-csv/#_transaction_batching)\n(a.k.a. batching), loading CSVs can be done in parallel, as well.\n### Creating `R` and CRAN Nodes\nAs all projects that are to be loaded are hosted on CRAN, the first\nnode to be created in the graph is the CRAN `Platform` node itself:\n```cypher\nCREATE (:Platform {name: 'CRAN'});\n```\nNot all projects hosted on CRAN are written in `R`, but those are\nthe focus of this analysis, so we need a `R` `Language` node:\n```cypher\nCREATE (:Language {name: 'R'});\n```\nWith these two, we create the first relationship of the graph:\n```cypher\nMATCH (p:Platform {name: 'CRAN'})\nMATCH (l:Language {name: 'R'})\nCREATE (p)-[:HAS_DEFAULT_LANGUAGE]-\u003e(l);\n```\nNow we can load the rest of the entities in our graph, connecting them\nto these as appropriate, starting with `Project`s.\n### Neo4j's `MERGE` Operation\nThe key operation when loading data to Neo4j is the\n[MERGE clause](https://neo4j.com/docs/cypher-manual/current/clauses/merge/#query-merge-using-unique-constraints).\nUsing the property specified in the query, MERGE either MATCHes the node/relationship\nwith the property, and, if it doesn't exist, duly CREATEs the node/relationship.\nIf the property in the query has a uniqueness constraint, Neo4j can thus iterate\nover possible duplicates of the \"same\" node/relationship, only creating it once,\nand \"attaching\" nodes to the uniquely-specified node on the go.\n\nThis is a double-edged sword, though, in the situation of creating relationships\nbetween unique nodes; if the participating nodes are not specified exactly, to\nMERGE a relationship between them will create __new__ node(s) that are duplicates.\nThis is undesirable from an ontological perspective, as well as a database\nefficiency perspective. So, all this to say that, to create unique\nnode-relationship-node entities requires _three_ passes over a CSV: the first\nto MERGE the first node type, the second to MERGE the second node type, and\nthe third to MATCH node type 1, MATCH node type 2, and MERGE the relationship\nbetween them.\n\nLastly, for the same reason as the above, it is necessary to create \"base\" nodes\nbefore creating nodes that \"stem\" from them. For example, if we had not created\nthe `R` `Language` node above (with unique property `name`), for every `R`\nproject MERGED from the projects CSV, Neo4j would create a new `Language` node\nwith name 'R' and a relationship between it and the R `Project` node.\nThis duplication can be useful in some data models, but in the interest of\nparsimony, we will load data in the following order:\n  1. `Project`s\n  2. `Version`s\n  3. Dependencies among `Project`s and `Version`s\n  4. `Contributor`s\n#### Loading `Project`s\nFirst up is the `Project` nodes. The source CSV for this type of node is\n[cran\\_projects.csv](https://github.com/ebb-earl-co/libraries_io_CRAN/blob/master/data/cran_subsetting.md#projects)\nand the queries are in\n[this file](https://github.com/ebb-earl-co/libraries_io_CRAN/blob/master/cypher/projects_apoc.cypher).\nNeo4j loads the CSVs data following the instructions of the file with the\n[`apoc.cypher.runFile`](https://neo4j.com/docs/labs/apoc/current/cypher-execution/) command; i.e.\n```\nCALL apoc.cypher.runFile('/path/to/libraries_io/cypher/projects_apoc.cypher') yield row, result return 0;\n```\nThe result of this set of queries is that the following portion of our graph\nis populated:\n\n![post-projects\\_apoc](images/projects_apoc-cypher_result.png)\n#### Loading `Version`s\nNext are the `Version`s of the `Project`s. The source CSV for this type\nof node is [cran\\_versions.csv](https://github.com/ebb-earl-co/libraries_io_CRAN/blob/master/data/cran_subsetting.md#versions)\nand the queries are in\n[this file](https://github.com/ebb-earl-co/libraries_io_CRAN/blob/master/cypher/versions_apoc.cypher).\nThese queries are run with\n```\nCALL apoc.cypher.runFile('/path/to/libraries_io/cypher/versions_apoc.cypher') yield row, result return 0;\n```\nThe result of this set of queries is that the graph has grown to include\nthe following nodes and relationships:\n\n![post-versions\\_apoc](images/versions_apoc-cypher_result.png)\n#### Loading Dependencies among `Project`s and `Version`s\nNow that there are `Project` nodes and `Version` nodes, it's time to\nlink their dependencies. The source CSV for these data is\n[cran\\_dependencies.csv](https://github.com/ebb-earl-co/libraries_io_CRAN/blob/master/data/cran_subsetting.md#dependencies)\nand this query is in\n[this file](https://github.com/ebb-earl-co/libraries_io_CRAN/blob/master/cypher/dependencies_apoc.cypher).\nBecause the `Project`s and `Version`s already exist, this operation\nis just the one MATCH-MATCH-MERGE query, creating relationships. It is run with\n```\nCALL apoc.cypher.runFile('/path/to/libraries_io/cypher/dependencies_apoc.cypher') yield row, result return 0;\n```\n##### Caveat\nDespite the Libraries.io Open Data dataset that contains dependencies among\n`R` projects, there are some projects that have no versions listed on CRAN,\nyet still report `imports` relationships on their CRAN sites. So, in order\nto include the impact of these `DEPENDS_ON` relationships in the degree\ncentrality algorithm, a pseudo `Version` node was created with the number\nproperty `\"NONE\"`, and an auto-generated UUID from the `apoc.create.uuid`\nfunction; i.e.\n```cypher\nCREATE (v:Version{name:\"NONE\", number: apoc.create.uuid()});\n```\nThen, the `R` script found\n[here](https://github.com/ebb-earl-co/libraries_io_CRAN/blob/master/get_missing_dependencies_from_crandb_api.R)\ncreates a JSON file using that `Version` node, attached to `Project` nodes with\nno `DEPENDS_ON` relationships in the current graph. Then, the JSON file\nis loaded to Neo4j using the Cypher query\n[here](https://github.com/ebb-earl-co/libraries_io_CRAN/blob/master/cypher/supplementary_dependencies.cypher).\nThat is, using the Neo4j cypher shell, and the `Rscript` executable from `R`:\n```bash\nGRAPHDBPASS=graph_db_pass_here Rscript --vanilla get_missing_dependencies_from_crandb_api.R \u003e some_file.json \u0026\u0026 bin/cypher-shell -u neo4j -p $GRAPHDBPASS\n```\nwhich opens up the cypher-shell in which the aforementioned Cypher query and\nthe just-created JSON file can be passed to the shell.\n\nThe result of these operations is that the graph has grown to include\nthe `DEPENDS_ON` relationship:\n\n![post-dependencies\\_apoc](images/dependencies_apoc-cypher_result.png)\n#### Loading `Contributor`s\nBecause the data corresponding to `R` `Project` `Contributor`s was\nretrieved from the Libraries.io API, it is not run with Cypher from a file, but\nin a Python script, particularly\n[this section](https://github.com/ebb-earl-co/libraries_io_CRAN/blob/master/python/merge_contributors.py#L127-L138).\n\n##### Caveat\nUnfortunately, that's not the end of the story for the `Contributor`\ndata: over 70% of the `R` `Project`s have no `Contributor`s reported\nby the Libraries.io API. So, even after the ~15k `Project` `Contributor`s\nwere scraped from the API, more than 10k of those needed `Contributor`\ndata imputed. To do this, I used the\n[`crandb`](https://github.com/r-hub/crandb#the-crandb-api) package\nfrom one of the Top-10 most-influential `Contributor`s, Gábor Csárdi.\nFor each package on CRAN, the `crandb` package will return the information\non its official CRAN page, in an `R` object that is easily parsed. For\nexample, using `crandb` on the venerable bootstrapping package, `boot`,\ngives `Contributor` in the form of Author and Maintainer:\n```r\n\u003e library(crandb)\n\u003e crandb::package('boot')\nCRAN package boot 1.3-23, 4 months ago\nTitle: Bootstrap Functions (Originally by Angelo Canty for S)\nMaintainer: Brian Ripley \u003cripley@stats.ox.ac.uk\u003e\nAuthor: Angelo Canty [aut], Brian Ripley [aut, trl, cre] (author of\n    parallel support)\n# ...\n```\nThe `Maintainer` field is always of the form \"Maintainer: name \u003cemail\u003e\",\nso that text was extracted and used as the `name` property of the\n`Contributor` node for the `Project`. The `Author` field proved to\nbe too unstructured for reliable scraping. This process is in\n[this `R` file](https://github.com/ebb-earl-co/libraries_io_CRAN/blob/master/impute_cran_packages_without_contributors.R).\n\nAfter executing this process, the graph is now in its final form:\n\n![post-merge\\_contributors](images/merge_contributors-py_result.png)\n## Preliminary Results\nOn the way to understanding the most influential `Contributor`,\nit is useful to find the most influential `Project`. Intuitively,\nthe most influential `Project` node should be the node with the\nmost (or very many) incoming `DEPENDS_ON` relationships; however,\nthe degree centrality algorithm is not as simple as just counting\nthe number of relationships incoming and outgoing and ordering by\ndescending cardinality (although that is a useful metric for\nunderstanding a [sub]graph). This is because the subgraph that\nwe are considering to understand the influence of `Project` nodes\nalso contains relationships to `Version` nodes.\n### Degree Centrality\nSo, using the Neo4j Graph Algorithm plugin's\n[`algo.degree`](https://neo4j.com/docs/graph-algorithms/current/algorithms/degree-centrality/#algorithms-degree-centrality)\nprocedure, all we need are a node label and a relationship type.\nThe arguments to this procedure could be as simple as two strings,\none for the node label, and one for the relationship type. However,\nas mentioned above, there are two node labels at play here, so we\nwill use the [alternative syntax](https://neo4j.com/docs/graph-algorithms/current/algorithms/degree-centrality/#algorithms-degree-cp)\nof the `algo.degree` procedure in which we pass Cypher statements\nreturning the set of nodes and the relationships among them.\n\nTo run the degree centrality algorithm on the `Project`s written\nin `R` that are hosted on `CRAN`, the syntax\n([found here](https://github.com/ebb-earl-co/libraries_io_CRAN/blob/master/cypher/project_degree_centrality.cypher))\nis:\n```cypher\ncall algo.degree(\n    \"MATCH (:Language {name:'R'})\u003c-[:IS_WRITTEN_IN]-(p:Project)\u003c-[:HOSTS]-(:Platform {name:'CRAN'}) return id(p) as id\",\n    \"MATCH (p1:Project)-[:HAS_VERSION]-\u003e(:Version)-[:DEPENDS_ON]-\u003e(p2:Project) return id(p2) as source, id(p1) as target\",\n    {graph: 'cypher', write: true, writeProperty: 'cran_degree_centrality'}\n)\n;\n```\n\nIt is **crucially** important to alias as `source` the `Project`\nnode MATCHed in the second query as the _end node_ of the\n`DEPENDS_ON` relationship, and the _start node_ of the\nrelationship as `target`. This is not officially documented,\nbut the example in the documentation has it as such, and I ran\ninto Java errors if not aliased exactly that way.\n\nNow that there is a property on each `R` `Project` node denoting its\ndegree centrality score, the following query returns the top 10 `Project`s:\n```cypher\nMATCH (:Language {name:'R'})\u003c-[:IS_WRITTEN_IN]-(p:Project)\u003c-[:HOSTS]-(:Platform {name:'CRAN'})\nRETURN p.name, p.cran_degree_centrality ORDER BY p.cran_degree_centrality DESC LIMIT 10\n;\n```\n\n|Project|Degree Centrality Score|\n|---|---|\n|`Rcpp`|6048|\n|`ggplot2`|4269|\n|`MASS`|4024|\n|`dplyr`|3573|\n|`plyr`|3017|\n|`stringr`|2622|\n|`Matrix`|2512|\n|`magrittr`|2200|\n|`httr`|2073|\n|`jsonlite`|2070|\n\nThe `Project` that is out in front by a good margin is `Rcpp`, the `R` package\nthat allows developers to integrate `C++` code into `R`, usually for significant\nspeedup. Another interesting note is that 4 of these top 10 are part of the \"Tidyverse\",\nHadley Wickham's collection of packages designed for data science. Moreover, as\nnoted on the [Tidyverse website](https://www.tidyverse.org/packages),\nthe last two `Project`s, `httr` and `jsonlite`, are \"Tidyverse-adjacent\", in that\nthey have a similar design and philosophy. It seems that the hypothesis that\n@hadley is the most influential contributor deserves a hefty amount of a priori weight!\n### The Most Influential Contributor\nTo properly evaluate the hypothesis, the degree centrality algorithm will be\nrun again, this time focusing on the `Contributor` nodes, and their contributions\nto `Project`s. The query\n([found here](https://github.com/ebb-earl-co/libraries_io_CRAN/blob/master/cypher/contributor_degree_centrality.cypher))\nis:\n```cypher\ncall algo.degree(\n    \"MATCH (:Platform {name:'CRAN'})-[:HOSTS]-\u003e(p:Project) with p MATCH (:Language {name:'R'})\u003c-[:IS_WRITTEN_IN]-(p)\u003c-[:CONTRIBUTES_TO]-(c:Contributor) return id(c) as id\",\n    \"MATCH (c1:Contributor)-[:CONTRIBUTES_TO]-\u003e(:Project)-[:HAS_VERSION]-\u003e(:Version)-[:DEPENDS_ON]-\u003e(:Project)\u003c-[:CONTRIBUTES_TO]-(c2:Contributor) return id(c2) as source, id(c1) as target\",\n    {graph: 'cypher', write: true, writeProperty: 'cran_degree_centrality'}\n)\n;\n```\nThis puts a property on each `Contributor` node denoting its\ndegree centrality score, and the following query returns the top 10 `Contributor`s and their scores:\n```cypher\nMATCH (:Platform {name:'CRAN'})-[:HOSTS]-\u003e(p:Project)-[:IS_WRITTEN_IN]-\u003e(:Language {name: 'R'})\nMATCH (c:Contributor)-[:CONTRIBUTES_TO]-\u003e(p)\nRETURN c.name, c.cran_degree_centrality ORDER BY c.cran_degree_centrality DESC LIMIT 10\n;\n```\n\n|Contributor|GitHub login|Degree Centrality Score|# Top-10 Contributions|# Total Contributions|Total Contributions Rank|\n|---|---|---|---|---|---|\n|Hadley Wickham|hadley|239 829|5|121|2nd|\n|Jim Hester|jimhester|167 662|3|120|3rd|\n|Kiril Müller|krlmlr|154 655|3|106|5th|\n|Jennifer (Jenny) Bryan|jennybc|119 082|3|57|13th|\n|Mara Averick|batpigandme|118 792|3|50|15th|\n|Hiroaki Yutani|yutannihilation|98 164|3|49|16th|\n|Christophe Dervieux|cderv|98 078|3|36|28th|\n|Gábor Csárdi|gaborcsardi|93 968|2|91|6th|\n|Jeroen Ooms|jeroen|72 211|2|117|4th|\n|Craig Citro|craigcitro|71 207|3|15|107th|\n\nAs was surmised from the result of the `Project`s degree centrality query, the\nmost influential `R` contributor on CRAN is Hadley Wickham, and it's not even close.\nNot only has does @hadley contribute to the second-most `R` projects of _any_\n`Contributor` (only behind `Contributor` Scott Chamberlain who is curiously\nabsent from the élité of most influential), he contributes to the most Top-10\nprojects of any `Contributor`, with fully half bearing his mark.\n\nThere are only 253 `Contributor`s who contribute to a Top-10 project–in terms\nof degree centrality–however even being one of those is not a sufficient\ncondition for a high degree centrality score; i.e. even though this table hints\nat a correlation between degree centrality score and number of total projects\n(query [here](https://github.com/ebb-earl-co/libraries_io_CRAN/blob/master/cypher/most_contributions_total.cypher)\nand rank query [here](https://github.com/ebb-earl-co/libraries_io_CRAN/blob/master/cypher/total_contributions_ranks.cypher))\ncontributed to, there is a higher association between degree centrality and\nnumber of _Top-10_ projects contributed to.\nIndeed, using the [`algo.similarity.pearson` function](https://neo4j.com/docs/graph-algorithms/current/experimental-algorithms/pearson/#algorithms-similarity-pearson-function-sample):\n```cypher\nMATCH (:Language {name:'R'})\u003c-[:IS_WRITTEN_IN]-(p:Project)\u003c-[:HOSTS]-(:Platform {name:'CRAN'})\nWITH p order by p.cran_degree_centrality DESC\nWITH collect(p) as r_projects\nUNWIND r_projects as project\nSET project.cran_degree_centrality_rank = apoc.coll.indexOf(r_projects, project)+1\nWITH project WHERE project.cran_degree_centrality_rank \u003c= 10\nMATCH (project)\u003c-[ct:CONTRIBUTES_TO]-(c:Contributor)\nWITH c, count(ct) as num_top_10_contributions\nWITH collect(c.cran_degree_centrality) as dc, collect(num_top_10_contributions) as tc\nRETURN algo.similarity.pearson(dc, tc) AS degree_centrality_top_10_contributions_correlation_estimate;\n```\nyields an estimate of 0.8462, whereas\n```cypher\nMATCH (:Language {name: 'R'})\u003c-[:IS_WRITTEN_IN]-(p:Project)\u003c-[:HOSTS]-(:Platform {name: 'CRAN'})\nMATCH (p)\u003c-[ct:CONTRIBUTES_TO]-(c:Contributor)\nWITH c, count(ct) as num_total_contributions\nWITH collect(c.cran_degree_centrality) as dc, collect(num_total_contributions) as tc\nRETURN algo.similarity.pearson(dc, tc) AS degree_centrality_total_contributions_correlation_estimate\n;\n```\nis only 0.6830.\nAll this goes to show that, in a network, the centrality of a\nnode is determined by contributing to the _right_ nodes,\nnot necessarily the _most_ nodes.\n## Conclusion\nUsing the Libraries.io Open Data dataset, the `R` projects\non CRAN and their contributors were analyzed using Neo4j–in\nparticular, the degree centrality algorithm–to find out which\ncontributor is the most influential to the graph of `R`\npackages, versions, dependencies, and contributors. That contributor\nis @hadley: the Tidyverse creator, Hadley Wickham.\n\nThis analysis did not take advantage of a commonly-used feature of\ngraph data; weights of the edges between nodes. A future improvement\nof this analysis would be to use the number of versions of a project,\nsay, as the weight in the degree centrality algorithm to down-weight\nthose projects that have few versions as opposed to the projects that\nhave verifiable \"weight\" in the `R` community, e.g. `dplyr`.\nSimilarly, it was not possible to delineate the type of contribution\nmade in this analysis; more accurate findings would no doubt result\nfrom the distinction between a package's author, for example, and a\ncontributor who merged a small pull request to fix a typo. Similarly,\nthe imputation of just a single contributor for more than 70% of the\n`R` packages potentially influenced in a non-trivial way the topology\nof this network.\n\nMoreover, the data used in this analysis are just a snapshot of the\nstate of CRAN from December 22, 2018: needless to say the number of\nversions and projects and contributions is always in flux and so\nbehooves updating. However, the Libraries.io Open Data are a good\nwindow into the dynamics of statistical programming's premier\ncommunity.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Febb-earl-co%2Flibraries_io_cran","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Febb-earl-co%2Flibraries_io_cran","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Febb-earl-co%2Flibraries_io_cran/lists"}