{"id":13557347,"url":"https://github.com/karlicoss/stexport","last_synced_at":"2025-04-03T02:30:48.775Z","repository":{"id":52507334,"uuid":"209131352","full_name":"karlicoss/stexport","owner":"karlicoss","description":"Export and access your Stackexchange data","archived":false,"fork":false,"pushed_at":"2025-03-11T00:41:55.000Z","size":32,"stargazers_count":11,"open_issues_count":3,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-02T20:05:09.352Z","etag":null,"topics":["backup","data-liberation","export","stackexchange","stackoverflow","takeout"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/karlicoss.png","metadata":{"files":{"readme":"README.org","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-09-17T18:42:59.000Z","updated_at":"2025-03-11T00:41:56.000Z","dependencies_parsed_at":"2023-01-19T00:45:47.153Z","dependency_job_id":null,"html_url":"https://github.com/karlicoss/stexport","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karlicoss%2Fstexport","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karlicoss%2Fstexport/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karlicoss%2Fstexport/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karlicoss%2Fstexport/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/karlicoss","download_url":"https://codeload.github.com/karlicoss/stexport/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246925196,"owners_count":20855853,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backup","data-liberation","export","stackexchange","stackoverflow","takeout"],"created_at":"2024-08-01T12:04:17.722Z","updated_at":"2025-04-03T02:30:48.738Z","avatar_url":"https://github.com/karlicoss.png","language":"Python","funding_links":[],"categories":["Python","others"],"sub_categories":[],"readme":"#+begin_src python :dir src :results drawer :exports results\nimport stexport.export as E; return E.make_parser().prog\n#+end_src\n\n#+RESULTS:\n:results:\nExport your personal Stackexchange data\n:end:\n\n* Setting up\n1. install with PIP\n   - =pip3 install --user git+https://github.com/karlicoss/stexport=\n\n   - for export functionality: append =[export]=\n   - for optional extras for logging and faster json processing: append =[optional]=\n   - or any combination of the above, e.g. =[export,optional]=\n\n   - alternatively, use =git clone --recursive=, or =git pull \u0026\u0026 git submodules update --init=. After that, you can use =pip3 install --editable=.\n\n2. Get API =key=\n   Register a new Oauth app [[https://meta.stackexchange.com/questions/261829/where-i-can-get-my-access-token-key-for-the-api][this]] for information.\n   Once you've registered, find your app in [[https://stackapps.com/apps/oauth][the list]] and take note of\n   - 'Key' -- this is your =key= parameter\n   - 'Client Id' and 'Oauth Domain/Redirect URI' -- this might be useful for the next step\n3. Get API =access_token=\n   This is optional, since in principle you can run the script without access token.\n   But some functionality won't work (e.g. =--user-sites=), and you might miss some of your private data, so it's recommended to do.\n\n   Visit the following link, but\n\n   - instead of =12345=, substitute 'Client Id' from step 3\n   - instead of =http://example.com=, substitute 'Redirect URI' from step 3\n\n   : https://stackoverflow.com/oauth/dialog?client_id=12345\u0026redirect_uri=http://example.com\u0026scope=private_info+no_expiry\n   # https://api.stackexchange.com/docs/authentication#scope\n   After that you should be authorised and redirected. Copy the =access_token= somewhere, you'll need it later.\n\n* Exporting\n\nNOTE: even if you decided not to use =access_token=, you still need to pass the parameter, just set it to empty string.\n# TODO ugh. it doesn't capture --all-sites or --user-sites or anything. ugh\n# really need a more literate approach somehow..\n\n#+begin_src python :dir src :results drawer :exports results\nimport stexport.export as E; return E.make_parser().epilog\n#+end_src\n\n#+RESULTS:\n:results:\n\nUsage:\n\n*Recommended*: create =secrets.py= keeping your api parameters, e.g.:\n\n\n: key = \"KEY\"\n: access_token = \"ACCESS_TOKEN\"\n\n\nAfter that, use:\n\n: python3 -m stexport.export --secrets /path/to/secrets.py\n\nThat way you type less and have control over where you keep your plaintext secrets.\n\n*Alternatively*, you can pass parameters directly, e.g.\n\n: python3 -m stexport.export --key \u003ckey\u003e --access_token \u003caccess_token\u003e\n\nHowever, this is verbose and prone to leaking your keys/tokens/passwords in shell history.\n\n\n\nI *highly* recommend checking exported files at least once just to make sure they contain everything you expect from your export. If not, please feel free to ask or raise an issue!\n\n:end:\n\n\n* Using data\n  \n#+begin_src python :dir src  :results drawer :exports results\nimport stexport.exporthelpers.dal_helper as D; return D.make_parser().epilog\n#+end_src\n\n#+RESULTS:\n:results:\n\nYou can use =stexport.dal= (stands for \"Data Access/Abstraction Layer\") to access your exported data, even offline.\nI elaborate on motivation behind it [[https://beepb00p.xyz/exports.html#dal][here]].\n\n- main usecase is to be imported as python module to allow for *programmatic access* to your data.\n\n  You can find some inspiration in [[https://beepb00p.xyz/mypkg.html][=my.=]] package that I'm using as an API to all my personal data.\n\n- to test it against your export, simply run: ~python3 -m stexport.dal --source /path/to/export~\n\n- you can also try it interactively: ~python3 -m stexport.dal --source /path/to/export --interactive~\n\n:end:\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkarlicoss%2Fstexport","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkarlicoss%2Fstexport","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkarlicoss%2Fstexport/lists"}