{"id":13557344,"url":"https://github.com/karlicoss/ghexport","last_synced_at":"2026-02-25T22:04:11.800Z","repository":{"id":43177576,"uuid":"77947610","full_name":"karlicoss/ghexport","owner":"karlicoss","description":"Export your Github activity: events, repositories, stars, etc.","archived":false,"fork":false,"pushed_at":"2026-01-31T21:22:28.000Z","size":60,"stargazers_count":57,"open_issues_count":1,"forks_count":7,"subscribers_count":3,"default_branch":"master","last_synced_at":"2026-02-15T11:58:30.952Z","etag":null,"topics":["backup","data-liberation","export","github","github-api","takeout"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/karlicoss.png","metadata":{"files":{"readme":"README.org","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2017-01-03T19:47:07.000Z","updated_at":"2026-02-02T22:36:21.000Z","dependencies_parsed_at":"2025-07-19T16:31:03.206Z","dependency_job_id":"0d15ff9b-1dfa-4c09-8ed8-f7d647d79ef2","html_url":"https://github.com/karlicoss/ghexport","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/karlicoss/ghexport","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karlicoss%2Fghexport","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karlicoss%2Fghexport/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karlicoss%2Fghexport/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karlicoss%2Fghexport/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/karlicoss","download_url":"https://codeload.github.com/karlicoss/ghexport/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karlicoss%2Fghexport/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29842872,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-25T21:18:31.832Z","status":"ssl_error","status_checked_at":"2026-02-25T21:18:29.265Z","response_time":61,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backup","data-liberation","export","github","github-api","takeout"],"created_at":"2024-08-01T12:04:17.605Z","updated_at":"2026-02-25T22:04:11.779Z","avatar_url":"https://github.com/karlicoss.png","language":"Python","funding_links":[],"categories":["Python","github-api"],"sub_categories":[],"readme":"#+begin_src python :dir src :results drawer :exports results\nimport ghexport.export as E; return E.make_parser().prog\n#+end_src\n\n#+RESULTS:\n:results:\nExport your Github personal data: issues, PRs, comments, followers and followings, etc.\n\n*Note*: this only deals with metadata. If you want a download of actual git repositories, I recommend using [[https://github.com/josegonzalez/python-github-backup][python-github-backup]].\n:end:\n\n* Setting up\n1. The easiest way is =pip3 install --user git+https://github.com/karlicoss/ghexport=.\n\n   Alternatively, use =git clone --recursive=, or =git pull \u0026\u0026 git submodule update --init=. After that, you can use =pip3 install --editable=.\n2. To use the API, you need to get a [[https://github.com/settings/tokens][personal access token]] from settings. Note that you need to use =repo= scope.\n\n \n* Exporting\n\n#+begin_src python :dir src :results drawer :exports results\nimport ghexport.export as E; return E.make_parser().epilog\n#+end_src\n\n#+RESULTS:\n:results:\n\nUsage:\n\n*Recommended*: create =secrets.py= keeping your api parameters, e.g.:\n\n\n: token = \"TOKEN\"\n\n\nAfter that, use:\n\n: python3 -m ghexport.export --secrets /path/to/secrets.py\n\nThat way you type less and have control over where you keep your plaintext secrets.\n\n*Alternatively*, you can pass parameters directly, e.g.\n\n: python3 -m ghexport.export --token \u003ctoken\u003e\n\nHowever, this is verbose and prone to leaking your keys/tokens/passwords in shell history.\n\n\nYou can also import ~ghexport.export~ as a module and call ~get_json~ function directly to get raw JSON.\n\n\nI *highly* recommend checking exported files at least once just to make sure they contain everything you expect from your export. If not, please feel free to ask or raise an issue!\n\n:end:\n\n** Extra export options\n- you can control specific data you want to export via ~--include~ option (see ~--help~ for available fields)\n\n  By default, all data will be included in the export.\n\n- you can include or exclude [[https://docs.github.com/en/rest/metrics/traffic][repository traffic]] data via ~--include-repos-traffic~ or ~--exclude-repos-traffic~.\n\n  Currently it's included by default.\n\n  You might want to exclude it if you have some issues with traffic API endpoint (it tends to be flakier than other endpoints).\n\n\n* API limitations\n\n*WARNING*: github API limits extent to which you can retrieve certain data, e.g. [[https://developer.github.com/v3/activity/events][events]] you can only get events from the past 90 days, and not more than 300 events.\n\nI *highly* recommend to export regularly and keep old exports. Easy way to achieve it is command like this: \n\n: python3 -m ghexport.export --secrets /path/to/secrets.py \u003e\"export-$(date -I).json\"\n\nOr, you can use [[https://github.com/karlicoss/arctee][arctee]] that automates this.\n\n\nTo get your older data past 90 days, you can request a [[https://github.com/settings/admin][manual export]] in your account settings.\n\n# TODO hmm, mention that dal.py can handle this?\n\n* Known Issues\n\nThe =requests= (and therefore =PyGithub=) modules on which this depends seems to sometimes fail to login if a =~/.netrc= file is present, see [[https://github.com/psf/requests/issues/5801#issuecomment-901610012][here]] for context.\n\n* Using the data\n  \n#+begin_src python :dir src  :results drawer :exports results\nimport ghexport.exporthelpers.dal_helper as D; return D.make_parser().epilog\n#+end_src\n\n#+RESULTS:\n:results:\n\nYou can use =ghexport.dal= (stands for \"Data Access/Abstraction Layer\") to access your exported data, even offline.\nI elaborate on motivation behind it [[https://beepb00p.xyz/exports.html#dal][here]].\n\n- main usecase is to be imported as python module to allow for *programmatic access* to your data.\n\n  You can find some inspiration in [[https://beepb00p.xyz/mypkg.html][=my.=]] package that I'm using as an API to all my personal data.\n\n- to test it against your export, simply run: ~python3 -m ghexport.dal --source /path/to/export~\n\n- you can also try it interactively: ~python3 -m ghexport.dal --source /path/to/export --interactive~\n\n:end:\n\nExample output:\n\n: Your events:\n: Counter({'PushEvent': 181,\n:          'WatchEvent': 27,\n:          'CreateEvent': 22,\n:          'IssueCommentEvent': 20,\n:          'PullRequestEvent': 15,\n:          'IssuesEvent': 5,\n:          'DeleteEvent': 5,\n:          'ForkEvent': 3,\n:          'PullRequestReviewCommentEvent': 1})\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkarlicoss%2Fghexport","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkarlicoss%2Fghexport","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkarlicoss%2Fghexport/lists"}