{"id":37703986,"url":"https://github.com/compnet/googleplusparser","last_synced_at":"2026-01-16T13:05:55.787Z","repository":{"id":3779224,"uuid":"4856584","full_name":"CompNet/GooglePlusParser","owner":"CompNet","description":"Java program allowing to retrieve the G+ network","archived":false,"fork":false,"pushed_at":"2019-07-21T13:34:23.000Z","size":3757,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-09-10T05:08:52.111Z","etag":null,"topics":["google-plus","network-extraction","social-networks"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"octonion/soccer","license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CompNet.png","metadata":{"files":{"readme":"readme.txt","changelog":null,"contributing":null,"funding":null,"license":"license.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2012-07-02T06:44:07.000Z","updated_at":"2021-10-18T07:48:51.000Z","dependencies_parsed_at":"2022-09-16T04:41:42.403Z","dependency_job_id":null,"html_url":"https://github.com/CompNet/GooglePlusParser","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/CompNet/GooglePlusParser","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CompNet%2FGooglePlusParser","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CompNet%2FGooglePlusParser/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CompNet%2FGooglePlusParser/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CompNet%2FGooglePlusParser/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CompNet","download_url":"https://codeload.github.com/CompNet/GooglePlusParser/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CompNet%2FGooglePlusParser/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28478913,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-16T11:59:17.896Z","status":"ssl_error","status_checked_at":"2026-01-16T11:55:55.838Z","response_time":107,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["google-plus","network-extraction","social-networks"],"created_at":"2026-01-16T13:05:54.810Z","updated_at":"2026-01-16T13:05:55.775Z","avatar_url":"https://github.com/CompNet.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"\nGoogle+ Network Extractor - version 1\n\n-----------------------------------------------------------------------\n\nCopyright 2011-2012 Vincent Labatut\nhttp:\\\\vincentlabatut.org\nvincent.labatut@free.fr\n\nGoogle+ Network Extractor is free software: you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation.\n\nFor source availability and license information see licence.txt\n\n-----------------------------------------------------------------------\n\nDescription: This software was designed to retrieve the social network\nbehind the Google+ service. It was designed during summer 2011 and tested\nstarting from september 2011. At this time, there was no official documented \nG+ API, so I had to search the web for related programs. I found a very nice\npiece of open source code by Jason Grey (cf. the parser package), which \nI used to access the relevant data: followers and followees. Note as of 2012,\nthe official Google+ API still does not document this feature (it focuses\non some personal data, by opposition to these relational ones).\nI also needed to obtain the complete list of G+ users. For this, I used\nsome ideas from Paul Anthony and NgocChinh (cf. the explorer package). \n\nHere is how the program works: first it scans some publicly available Google \nfiles to retrieve all the Google ids (gid), which are unique 21 digits\nnumbers associated to each Google user (including people actually not using\nGoogle+). Those are inserted in a Person table located in a H2 database.\nSecond, for each of these gid, it retrieves the lists of followers\nand followees, also under the form of gid. These are inserted in a Relationship\ntable. To speed up the process, this part is multithreaded. Third, once all \nthe data has been gathered, it exports the social network (basically the \nrelationship table) under the form of an edgelist file (list of links, each \none defined by the source and target nodes).\n\nThe program was tested first to retrieve my ego-network (i.e. the people\naround me, and those around them, etc.). Once the program was working, I \nstarted retrieving the whole G+ network. It took much longer than I thought\n(around 2 months), for a network containing 80,080,893 nodes and 473,106,758 \nlinks (not counting isolates). I initially had planned to retrieve the network \nevery two months to track its evolution. However, Google eventually added a\ncaptcha system, preventing me from reaching this goal (I started retrieving \nthe November network, but could not finish). Finally, I decided to focus on \nwhat I had, and to just analyze the Sepbember-October network. The result is \nthe following paper, which I ask you to cite if you use this software for \nacademic purposes:\n\u003cTODO to be completed\u003e \n\nBesides the functionalities exposed here, the software can also retrieve\nother data from a gid, such as name, firstname, etc. I eventually did not\nretrieve these because first I did not really need them, and second it was\nmaking the whole process extremely long. \n\nMoreover, I implemented some additional post-processing, partly in R and \npartly in Java. Indeed, the R iGraph library is very convenient to process\ntopological properties, but some of its functions did not work on such a\nlarge network (at least on my computer), for memory matters. So I had to\ncode a part of the post processing in Java.\nThe res/rscript folder contains the following R scripts:\n  - edgelist2graphml.R is able to convert an edgelist file to a graphml file, \n    so that you can play with it in Gephi (http://gephi.org/), for instance. \n    The script requires the iGraph library v0.5 (http://igraph.sourceforge.net/). \n    Note all those softwares are open source.\n  - extractGiantComponent.R processes some properties on the whole network,\n    and extract the list of nodes corresponding to the giant component of the\n    whole network. This list must then be processed by the Java class SubgraphExtractor\n    in order to get the actual subgraph, represented as an edgelist.\n  - giantcompAnalysis.R is in charge of processing the properties of the\n    giant component (i.e. the file generated by the Java class).\n\nIt is important knowing the captcha system does not completely prevent using\nthis software: you just cannot do too many requests from the same IP. It considerably\nslows the process down, too much for my academic purposes... but it still is enough\nto play with it a bit. Google might also raise this limit, one day, or provide\nan academic access (which is not the case for this service, up to now).\n\nGoogle might also change the structure of its JSON objects, or the way its (undocumented)\nAPI is invoked, which would make the parser obsolete. Note I do not plan maintaining\nthis software and updating it if such changes were to happen. Please, do not contact\nme for these matters. I think the Javadoc is complete enough to allow anyone adapting\nthe program to his needs. I would be interested to be informed of any use/modification,\nthough! \n\nIf you plan to use this software to retreive the whole G+ network, note it\nhas an important limitation: gid are stored as string in the DB. Why strings\nand not integers, since they are actually integers, you might ask? Well, the\nDB was not able to store such large integers. A re-numbering of the modes\nwould have allowed to use much smaller values than those bloody 21 digits,\nbut I was not aware of that at this time. It turns out using strings was a\nhuge problem when I started processing the network (because of all the\nmemory it requires, of course). So I eventually had to renumber the nodes,\nwhich took quite a while. I did not update the program though, since I did not\nplan to use it again. In other words, this renumbering was performed completely\nas a post-processing. The program still retrieves and store strings.\n\n-----------------------------------------------------------------------\n\nThis product uses open source softwares:\n  + H2 - Java SAL Database\n    http://www.h2database.com\n    Mozilla Public License\n    Used to store the data retrived from Google+\n\t\n  + Jersey (Sun/Oracle implementation of JAX-RS - Java API for RESTful Web Services)\n    http://jersey.java.net/\n    GPL2 License\n    Used to query the Google+ service\n    \n  + Jettison - JSON StAX Implementation\n    http://jettison.codehaus.org/\n    Apache License v2 \n    Used to handle the JSON objects\n    \n  + Log4j\n    http://logging.apache.org/log4j\n    Apache License v2 \n    Used to log the program activity\n  \t\n  + Some ideas from these blogs\n    Paul Anthony: http://blog.webdistortion.com/2011/06/12/google-people-search-how-big-is-googles-social-network/\n    NgocChinh: http://www.blackhatworld.com/blackhat-seo/black-hat-seo/315777-list-google-profiles-do-what-you-want.html\n    Used to obtain all the Google+ ids\n\n  + Some source code from Jason Grey in the following classes:\n    GoogleJSONFilter\n    GooglePlusParser\n    https://plus.google.com/114074126238467553553/posts\n    (A G+ post, incidentally!)  \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcompnet%2Fgoogleplusparser","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcompnet%2Fgoogleplusparser","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcompnet%2Fgoogleplusparser/lists"}