{"id":20687042,"url":"https://github.com/itaru2622/bluesky-selfhost-env","last_synced_at":"2026-06-08T07:34:31.724Z","repository":{"id":231288191,"uuid":"764535635","full_name":"itaru2622/bluesky-selfhost-env","owner":"itaru2622","description":"bluesky self-hosting tool for easy deploy in anywhere.","archived":false,"fork":false,"pushed_at":"2026-06-06T09:38:16.000Z","size":1023,"stargazers_count":122,"open_issues_count":0,"forks_count":14,"subscribers_count":3,"default_branch":"master","last_synced_at":"2026-06-06T11:16:01.986Z","etag":null,"topics":["appview","atproto","bgs","bluesky","bluesky-social","bsky","hosting","labeler","ozone","pds","plc","self-host","self-hosted","self-hosting"],"latest_commit_sha":null,"homepage":"","language":"Makefile","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/itaru2622.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-02-28T08:58:31.000Z","updated_at":"2026-05-30T13:03:01.000Z","dependencies_parsed_at":"2024-05-12T01:25:00.899Z","dependency_job_id":"aafb1acf-bf16-40e3-8290-f9a608335ab1","html_url":"https://github.com/itaru2622/bluesky-selfhost-env","commit_stats":null,"previous_names":["itaru2622/bluesky-selfhost-env"],"tags_count":128,"template":false,"template_full_name":null,"purl":"pkg:github/itaru2622/bluesky-selfhost-env","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/itaru2622%2Fbluesky-selfhost-env","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/itaru2622%2Fbluesky-selfhost-env/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/itaru2622%2Fbluesky-selfhost-env/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/itaru2622%2Fbluesky-selfhost-env/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/itaru2622","download_url":"https://codeload.github.com/itaru2622/bluesky-selfhost-env/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/itaru2622%2Fbluesky-selfhost-env/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34053435,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-08T02:00:07.615Z","response_time":111,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["appview","atproto","bgs","bluesky","bluesky-social","bsky","hosting","labeler","ozone","pds","plc","self-host","self-hosted","self-hosting"],"created_at":"2024-11-16T22:55:35.848Z","updated_at":"2026-06-08T07:34:31.718Z","avatar_url":"https://github.com/itaru2622.png","language":"Makefile","funding_links":[],"categories":["Makefile"],"sub_categories":[],"readme":"# \u003ca id=\"top\"/\u003eself-hosting entire bluesky\n\nhttps://github.com/itaru2622/bluesky-selfhost-env\n\n## Contents:\n  - [Motivation](#motivation)\n  - [Current Status](#status)\n  - [Operations for self-hosting bluesky](#ops)\n      - [configuration](#ops0-configparams)\n      - [prepare your network](#ops1-prepare)\n      - [check](#ops2-check)\n      - [deploy](#ops3-run)\n      - [play](#ops5-play)\n      - [shutdown](#ops6-stop)   \n  - [Hacks](#hack)\n      - [Create accounts on your bluesky in easy](#hack-ops-CreateAccount)\n      - [Build from source by yourself](#hack-clone-and-build)\n      - Check Env Vars [in docker-compose](#hack-EnvVars-Compose) and [in sources](#hack-EnvVars-Sources)\n      - [Create a table showing {env x container =\u003e value} from source and docker-compose](#hack-EnvVars-Table)\n  - [Appendix](#appendix)\n      - [Screen shots](#screenshots)\n      - [Sourses in Use](#sources)\n      - [Sample DNS Server Config(bind9)](#sample-dns-config)\n  - [References](#refs)\n\n## \u003ca id=\"motivation\" /\u003eMotivation\n\nThis repository aims to get self-hosted a bluesky environment easy, with:\n\n - Configurable hosting domain: easily tuned by environment variable (${DOMAIN}).\n - Reproducibility: full disclosure of all configurations and operations, including reverse proxy rules and patches to the original code of bluesky-social.\n - Simplicity: all bluesky components run on one host, powered by docker-compose.\n - Minimal remapping: the simplest possible mapping rules between FQDN, reverse proxy, and docker-container, for easy understanding and tuning.\n\nCurrently, my latest release is \u003cstrong\u003e2026-05-30\u003c/strong\u003e, based on the \u003cstrong\u003e2026-05-30\u003c/strong\u003e code from bluesky-social.\u003cbr\u003e\n\n### Special notes about big impact changes in upstream regarding selfhost\n\n- changes in end of Jan 2026, bgs(indigo/cmd/bgs) has gone and replaced to relay(indigo/cmd/relay) which requires PublicOnly scenario.\n  It breaks scenarios using privateIP, development etc. so the scenarios are recovered by involving new runtime env var (INDIGO_SSRF_PUBLIC_ONLY_SCENARIO).\n  It is allowed to use privateIP, only when INDIGO_SSRF_PUBLIC_ONLY_SCENARIO=false is set.\n  to achive it, indigo got patch in util/ssrf/ssrf.go and docker-compose*.yaml.\u003cbr\u003e\n  It is safe to define INDIGO_SSRF_PUBLIC_ONLY_SCENARIO=false in all components regarding indigo repo becuase indigo/util/ssrf may be involved by any in indigo.\n- changes in Aug-Sep 2025, atproto-proxy(bluesky proxy header) is required by social-app, which value can tune only at build time.\n  It breaks 'build once, run with any domain' manner on this tool, and the manner is recovered by involving local static CDN in social-app container.\n  this technique is inspired from STATIC_CDN_HOST in social-app(bskyweb) codes.\n  to achieve it, social-app got some patches for bskyweb + Dockerfile + additional shell command and docker-compose*.yaml.\u003cbr\u003e\n   - in docker-compose-build.yaml, EXPO_PUBLIC_BLUESKY_PROXY_DID=@@EXPO_PUBLIC_BLUESKY_PROXY_DID@@ as build arg, to embed placeholder(mark) within js files.\n   - on booting phase at runtime, it rewrites placeholder with real val by runtime env EXPO_PUBLIC_BLUESKY_PROXY_DID=did:web:..., before being used by social-app.\n   - refer comments in docker-compose.yaml for detail.\n\n## \u003ca id=\"status\"/\u003eCurrent status regarding self-hosting\n\nAs shown below, most features work as expected in the self-hosting environment.\u003cbr\u003e\nUnfortunately, some features may not work correctly; the reasons for this are described in https://github.com/bluesky-social/atproto/discussions/2334\u003cbr\u003e\n\nTest results with 'asof-2024-06-02' and later:\u003cBR\u003e\n\n   -  ok: Create account on pds (via social-app, bluesky API).\n   -  ok: Basic usages on social-app\n       -  ok: Sign in, edit profile, post/repost articles, search posts/users/feeds, vote like/follow.\n       -  ok: Receive notifications when others vote like/follow you.\n       -  ok: Subscribe/unsubscribe to labeler in profile page.\n       -  ok: Report to labeler for any post.\n       -  not yet: DM(chat) with others.\n   -  ok: Integrate with [feed-generator](https://github.com/bluesky-social/feed-generator) NOTE: it has some delay, reload on social-app.\n   -  ok: Moderate with [ozone](https://github.com/bluesky-social/ozone).\n       -  ok: Sign in and configure labels on ozone-UI.\n       -  ok: Receive the report sent by user.\n       -  ok: Assign label to the post/account on ozone-UI, then events published to subscribeLabels.\n       -  ok: The view of post changes on social-app when using [workaround tool](https://github.com/itaru2622/bluesky-selfhost-env/blob/master/ops-helper/apiImpl/subscribeLabels2BskyDB.ts).\n          -  NOTE: without workaround tool, the view is not changed. refer https://github.com/bluesky-social/atproto/issues/2552\n   -  ok: Subscribe to events from pds/bgs(relay)/ozone by firehose/websocket.\n   -  ok: Subscribe to events from jetstream, since 2024-10-19r1\n   -  not yet: Others.\n\n[back to top](#top)\n## \u003ca id=\"ops\"/\u003eOperations for self-hosting bluesky (powered by Makefile)\n\nThe following operations assume that the self-hosting domain is \u003cstrong\u003emysky.local.com\u003c/strong\u003e (defined in Makefile).\u003cbr\u003e\nYou can change the domain name by setting the environment variable as follows:\n\n### \u003ca id=\"ops0-configparams\"/\u003e0) Configure params and install tools for ops\n\n```bash\n### \u003ca id=\"ops0-configparams\"/\u003e0) Configure parameters and install tools\n\n```bash\n# 1) Set domain name for self-hosting bluesky\nexport DOMAIN=whatever.yourdomain.com\n\n# 2) Set 'asof' date (YYYY-MM-DD or 'latest') to select docker images and sources.\n#    Example: 2026-05-30 (latest prebuild) or 'latest' (following docker image naming).\nexport asof=2026-05-30\n\n# 3) Set email addresses:\n\n# 3-1) EMAIL4CERTS: for Let's Encrypt certificate signing.\nexport EMAIL4CERTS=your@mail.address\n# Use 'internal' (reserved) for self-signed certificates to avoid rate limits during setup.\nexport EMAIL4CERTS=internal\n\n# 3-2) PDS_EMAIL_SMTP_URL: for PDS (e.g., smtps://youraccount:your-app-password@smtp.gmail.com)\nexport PDS_EMAIL_SMTP_URL=smtps://\n\n# 3-3) FEEDGEN_EMAIL: for feed-generator account.\nexport FEEDGEN_EMAIL=feedgen@example.com\n\n## Install required tools (if missing).\napt install -y make pwgen\n(cd ops-helper/apiImpl ; npm install)\n(sudo curl -o /usr/local/bin/websocat -L https://github.com/vi/websocat/releases/download/v1.14.1/websocat.x86_64-unknown-linux-musl; sudo chmod a+x /usr/local/bin/websocat)\n\n# 4) Check configuration.\nmake echo\n\n# 5) Generate and check container secrets.\nmake genSecrets\n```\n\n### \u003ca id=\"ops1-prepare\"/\u003e1) Prepare your network\n\n1) Create DNS A-Records in your self-hosting network.\u003cBR\u003e\n\nAt a minimum, you will need the following two A-Records.\u003cBR\u003e\nRefer the [appendix](#sample-dns-config) for a sample DNS server (bind9) configuration.\n\n```\n     -    ${DOMAIN}\n     -  *.${DOMAIN}\n```\n\n2) Generate and install a CA certificate (necessary for private/closed networks and when working with self-signed certificates).\n    -  Once generated, copy the crt and key files to ./certs/root.{crt,key}\n    -  Important: Install root.crt on your host machine and within your browser.\nFollow the steps below to easily obtain self-signed CA certificates:\n\n```\n# Get and store the self-signed CA certificate into ./certs/root.{crt,key} with caddy.\nmake getCAcert\n# Install the CA certificate on the host machine.\nmake installCAcert\n\n# Remember to install the certificate in your browser.\n```\n\n### \u003ca id=\"ops2-check\"/\u003e2) Check if it's ready to self-host bluesky\n\n```bash\n# Check DNS server responses for your self-hosting domain\ndig  ${DOMAIN}\ndig  any.${DOMAIN}\n\n# Check if DNS works as expected. Test from all nodes you want to access your self-hosting bluesky, including host and client machines.\nping ${DOMAIN}\nping any.${DOMAIN}\n\n# Start containers for testing\nmake    docker-start f=./docker-compose-debug-caddy.yaml services=\n\n# Test HTTPS and WSS with your docker environment\ncurl -L https://test-wss.${DOMAIN}/\nwebsocat wss://test-wss.${DOMAIN}/ws\n\n# Test reverse proxy mapping to ensure it works as expected for bluesky\n#  These should redirect to PDS\ncurl -L https://pds.${DOMAIN}/xrpc/any-request | jq\ncurl -L https://some-hostname.pds.${DOMAIN}/xrpc/any-request | jq\n\n#  These should redirect to social-app\ncurl -L https://pds.${DOMAIN}/others | jq\ncurl -L https://some-hostname.pds.${DOMAIN}/others | jq\n\n# Stop test containers, without persisting data\nmake    docker-stop-with-clean f=./docker-compose-debug-caddy.yaml\n```\n\n=\u003e If testOK, then go ahead; otherwise, examine your environment.\n\n### \u003ca id=\"ops3-run\"/\u003e3) Deploy bluesky\n\nThis section first outlines deploying bluesky with prebuilt images.\u003cBR\u003e\nRefer [later](#hack-clone-and-build) for instructions on building images from sources independently.\n\n```bash\n# 0) Pull prebuilt docker images from docker.io to explicitly avoid building images.\nmake docker-pull\n\n# 1) Deploy the essential containers (database, caddy, etc.).\nmake docker-start\n\n# Wait for log messages to cease.\n\n# 2) Deploy the core bluesky containers (plc, bgs, appview, pds, ozone, ...).\nmake docker-start-bsky\n\n# The operation below is obsolete due to patching/152-indigo-newpds-dayper-limit.diff\n# 3) Configure the bgs parameter for the perDayLimit setting using the REST API.\n# ~~~ make api_setPerDayLimit ~~~\n```\n\n### \u003ca id=\"ops4-run-fg\"/\u003e4) Deploy the Feed Generator\n\n```bash\n# 1) Verify that the social-app is ready to serve content.\ncurl -L https://social-app.${DOMAIN}/\n\n# 2) Generate an account specifically for the feed generator.\nmake api_CreateAccount_feedgen\n\n# 3) Launch the bluesky feed-generator.\nmake docker-start-bsky-feedgen  FEEDGEN_PUBLISHER_DID=did:plc:...\n\n# 4) Publish the feed's existence (using scripts/publishFeedGen.ts on the feed-generator).\nmake publishFeed\n```\n\n### \u003ca id=\"ops4-run-ozone\"/\u003e4-2) Deploy Ozone\n\n```bash\n# 1) Generate an account for the ozone service or administrator.\n#  A working email address is essential, as ozone/PDS will send a confirmation code to it.\nmake api_CreateAccount_ozone                    email=your-valid@email.address.com handle=...\n\n# 2) Launch Ozone\n# Ozone uses the same DID for both OZONE_SERVER_DID and OZONE_ADMIN_DIDS, as documented in [HOSTING.md](https://github.com/bluesky-social/ozone/blob/main/HOSTING.md)\nmake docker-start-bsky-ozone  OZONE_SERVER_DID=did:plc:  OZONE_ADMIN_DIDS=did:plc:\n\n# 3) Run the workaround tool to index label assignments into the appview DB through subscribeLabels.\n# ./ops-helper/apiImpl/subscribeLabels2BskyDB.ts --help\n./ops-helper/apiImpl/subscribeLabels2BskyDB.ts\n\n# 4) [Required occasionally] Refresh the DidDoc prior to ozone sign-in (required since asof-2024-07-05)\n#    First, request and get PLC signature by email\nmake api_ozone_reqPlcSign                       handle=... password=...\n#    Then, update the didDoc using obtained signature\nmake api_ozone_updateDidDoc   plcSignToken=     handle=...  ozoneURL=...\n\n# 5) [Optional] Invite a new member to the ozone team (by assigning a role):\n#    Valid roles are: tools.ozone.team.defs#roleAdmin | tools.ozone.team.defs#roleModerator | tools.ozone.team.defs#roleTriage\nmake api_ozone_member_add   role=  did=did:plc:\n```\n\n### \u003ca id=\"ops4-run-jetstream\"/\u003e4-3) Deploy Jetstream\n```bash\nmake docker-start-bsky-jetstream\n```\n\n\n### \u003ca id=\"ops5-play\"/\u003e5) Play with self-hosted blusky.\n\nAccess ```https://social-app.${DOMAIN}/``` (e.g., ```https://social-app.mysky.local.com/```) in your browser.\n\nSee the [screenshots](./docs/screenshots) for instructions on creating or signing in to an account.\n\n### \u003ca id=\"ops5-play-jetstream\"/\u003e5-1) Subscribe Jetstream\n\n```bash\n# Subscribe almost all collections from jetstream\nwebsocat \"wss://jetstream.${DOMAIN}/subscribe?wantedCollections=app.bsky.actor.profile\u0026wantedCollections=app.bsky.feed.like\u0026wantedCollections=app.bsky.feed.post\u0026wantedCollections=app.bsky.feed.repost\u0026wantedCollections=app.bsky.graph.follow\u0026wantedCollections=app.bsky.graph.block\u0026wantedCollections=app.bsky.graph.muteActor\u0026wantedCollections=app.bsky.graph.unmuteActor\"\n```\n\n### \u003ca id=\"ops5-play-ozone\"/\u003e5-2) Play with ozone (moderation)\n\nAccess ```https://ozone.${DOMAIN}/configure``` (e.g., ```https://ozone.mysky.local.com/configure```) in your browser.\n\n### \u003ca id=\"ops6-stop\"/\u003e6) Stop all containters\n\n```bash\n# Choice 1: Shut down containers, retaining data.\nmake docker-stop\n\n# Choice 2: Shut down containers and delete the data.\nmake docker-stop-with-clean\n```\n\n[back to top](#top)\n## \u003ca id=\"hack\"/\u003eHack\n\n### \u003ca id=\"hack-ops-CreateAccount\"/\u003eCreate accounts on your bluesky easily\n\n```bash\nexport u=foo\nmake api_CreateAccount handle=${u}.pds.${DOMAIN} password=${u} email=${u}@example.com resp=./data/accounts/${u}.secrets\n\n# To create more accounts, simply re-assign $u and call the above operation, as shown below.\nexport u=bar\n!make\n\nexport u=baz\n!make\n```\n\n### \u003ca id=\"hack-clone-and-build\"/\u003eBuild docker images from sources yourself\n\nAfter configuring the [parameters](#ops0-configparams) and [optional environment variable](#hack-ops-development), proceed as follows:\n\n```bash\n# Get source code from all repositories\nmake    cloneAll\n\n# Create work branches and stay on them for all repositories (repos/*); optional but recommended for safety.\nmake    createWorkBranch\n```\n\nThen, build the docker images as follows:\n\n```bash\n# 0) Apply the minimum necessary patch to build images, regardless of self-hosting.\n#    See https://github.com/bluesky-social/atproto/discussions/2026 for details, specifically for feed-generator/Dockerfile etc.\n# NOTE: This operation will create a new branch, apply the patch, and stay on that new branch.\nmake patch-dockerbuild\n\n# 1) Build the images\nmake build DOMAIN= f=./docker-compose-builder.yaml\n```\n\n[back to top](#top)\n### \u003ca id=\"hack-ops-development\"/\u003eStreamlined Development Using Your Remote Fork Repository\n\nBy setting the fork_repo_prefix variable before cloneAll, it registers your remote fork repository with `git remote add fork ....`\nthen you have additional easy operations against multiple repositores, as below.\n\n```bash\nexport fork_repo_prefix=git@github.com:YOUR_GITHUB_ACCOUNT/\n\nmake cloneAll\n\n# Easily manage (push and pull) branches and tags for all repositories with a single command targeting your remote fork repositories.\nmake exec under=./repos/* cmd='git push fork branch'\nmake exec under=./repos/* cmd='git tag -a \"asof-XXXX-XX-XX\" '\nmake exec under=./repos/* cmd='git push fork --tags'\n\n# Push your develop-branch in justOneRepo working folder to your remote fork repository.\nmake exec under=./repos/justOneRepo cmd='git push fork develop-branch'\n\n# See the Makefile for complete details and usage examples.\n```\n\n[back to top](#top)\n### \u003ca id=\"hack-EnvVars-Compose\"/\u003eCheck Env Vars in docker-compose\n\n1) Get all env vars in docker-compose\n\n```bash\n# Names and their values\n_yqpath='.services[].environment, .services[].build.args'\n_yqpath='.services[].environment'\n\n# List of var=val\ncat ./docker-compose-builder.yaml | yq -y \"${_yqpath}\" \\\n  | grep -v '^---' | sed 's/^- //' | sort -u -f\n\n# Output in yaml\ncat ./docker-compose-builder.yaml | yq -y \"${_yqpath}\" \\\n  | grep -v '^---' | sed 's/^- //' | sort -u -f  \\\n  | awk -F= -v col=\":\" -v q=\"'\" -v sp=\"  \" -v list=\"-\" '{print   sp list sp q $1 q col sp q $2 q}' \\\n  | sed '1i defs:' | yq -y\n\n\n# List of names\ncat ./docker-compose-builder.yaml | yq -y \"${_yqpath}\" \\\n  | grep -v '^---' | sed 's/^- //' | sort -u -f \\\n  | awk -F= '{print $1}' | sort -u -f\n```\n\n2) Get env vars regarding {URL | DID | DOMAIN} == mapping rules in docker-compose\n\n```bash\n# get {name=value} of env vars regarding { URL | DID | DOMAIN }\ncat ./docker-compose-builder.yaml | yq -y .services[].environment \\\n | grep -v '^---' | sed 's/^- //' | sort -u -f \\\n | grep -e :// -e did: -e {DOMAIN}\n\n# get names of env vars regarding { URL | DID | DOMAIN }\ncat ./docker-compose-builder.yaml | yq -y .services[].environment \\\n | grep -v '^---' | sed 's/^- //' | sort -u -f \\\n | grep -e :// -e did: -e {DOMAIN} \\\n | awk -F= '{print $1}' | sort -u -f \\\n | tee /tmp/url-or-did.txt\n```\n\n3) Get mapping rules in reverse proxy (caddy )\n\n```bash\n# dump rules, no idea to convert into  easy readable format...\ncat config/caddy/Caddyfile\n```\n\n[back to top](#top)\n### \u003ca id=\"hack-EnvVars-Sources\"/\u003eCheck Env Vars in sources\n\n1) Get files related env vars in sources\n\n```bash\n# Files named *env*\nfind repos -type f | grep -v -e /.git/  | grep -i env \\\n  | grep -v -e .jpg$ -e .ts$  -e .json$ -e .png$ -e .js$\n\n# Files containing 'export'\nfind repos -type f | grep -v /.git/  | xargs grep -l export \\\n  | grep -v -e .js$ -e .jsx$  -e .ts$ -e .tsx$ -e .go$ -e go.sum$ -e go.mod$ -e .po$ -e .json$ -e .patch$ -e .lock$ -e .snap$\n```\n\n2) Get all env vars from source code\n\n```bash\n# In an easy way\n_files=repos\n# Ensure files to search for envs\n_files=`find repos -type f | grep -v -e '/.git' -e /__  -e /tests/ -e _test.go -e /interop-test-files  -e /testdata/ -e /testing/ -e /jest/ -e /node_modules/ -e /dist/ | sort -u -f`\n\n# For JavaScripts families, get env vars from process.env.ENVNAME\ngrep -R process.env ${_files} \\\n  | cut -d : -f 2- | sed 's/.*process\\.//' | grep '^env\\.' | sed 's/^env\\.//' \\\n  | sed -r 's/(^[A-Za-z_0-9\\-]+).*/\\1/' | sort -u -f \\\n  | tee /tmp/vars-js1.txt\n\n# For JavaScripts families, get env vars from envXXX('MORE_ENVNAME'), Refer to atproto/packages/common/src/env.ts for envXXX\ngrep -R -e envStr -e envInt -e envBool -e envList ${_files} \\\n  | cut -d : -f 2- \\\n  | grep -v -e ^import -e ^export -e ^function  \\\n  | sed \"s/\\\"/'/g\" \\\n  | grep \\' | awk -F\\' '{print $2}' | sort -u -f \\\n  | tee /tmp/vars-js2.txt\n\n# For golang, get env vars from EnvVar(s): []string{\"ENVNAME\", \"MORE_ENVNAME\"}\ngrep -R EnvVar ${_files} \\\n  | cut -d : -f 3- | sed -e 's/.*string//' -e 's/[,\"{}]//g' \\\n  | tr ' ' '\\n' | grep -v ^$ | sort -u -f \\\n  | tee /tmp/vars-go.txt\n\n# for docker-compose, get env vars from services[].environment\necho {$_files} \\\n  | tr ' ' '\\n' | grep -v ^$ | grep -e .yaml$ -e .yml$ | grep compose \\\n  | xargs yq -y .services[].environment | grep -v ^--- | sed 's/^- //' \\\n  | sed 's/: /=/' | sed \"s/'//g\" \\\n  | sort -u -f \\\n  | awk -F= '{print $1}' | sort -u -f \\\n  | tee /tmp/vars-compose.txt\n\n\n# Get unique lists\ncat /tmp/vars-js1.txt /tmp/vars-js2.txt /tmp/vars-go.txt /tmp/vars-compose.txt | sort -u -f \u003e /tmp/envs.txt\n\n# Pick env vars related to mapping {URL, ENDPOINT, DID, HOST, PORT, ADDRESS}\ncat /tmp/envs.txt  | grep -e URL -e ENDPOINT -e DID -e HOST -e PORT -e ADDRESS\n```\n\n3) Find {URL | DID | bsky } near env names in sources\n\n```bash\nfind repos -type f | grep -v -e /.git  -e __ -e .json$ \\\n  | xargs grep -R -n -A3 -B3 -f /tmp/envs.txt \\\n  | grep -A2 -B2 -e :// -e did: -e bsky\n```\n\n4) Find bsky.{social, app, network} in sources (to check hard-coded domain/FQDN)\n\n```bash\nfind repos -type f | grep -v -e /.git -e /tests/ -e /__ -e Makefile -e .yaml$ -e .md$  -e .sh$ -e .json$ -e .txt$ -e _test.go$ \\\n  | xargs grep -n -e bsky.social -e bsky.app -e bsky.network  -e bsky.dev\n```\n\n[back to top](#top)\n### \u003ca id=\"hack-EnvVars-Table\"/\u003eCreate a table showing {env x container =\u003e value} from source and docker-compose.\n\nThis task uses the result(/tmp/envs.txt) of [the above](#hack-EnvVars-Sources) as input.\n\n```bash\n# Create table showing { env x container =\u003e value } with the ops-helper script.\ncat ./docker-compose-builder.yaml | ./ops-helper/compose2envtable/main.py -l /tmp/envs.txt -o ./docs/env-container-val.xlsx\n```\n\n[back to top](#top)\n### \u003ca id=\"hack-self-signed-certs\"/\u003eRegarding self-signed certificates x HTTPS x containers.\n\nThis self-hosting env tries to use self-signed certificates as trusted certificates by installing them into containers.\nThe expected behavior is that by sharing /etc/ssl/certs/ca-certificates.crt amang all containers, containers can distinguish that those in ca-certificates.crt are trusted.\n\nUnfortunately, this approach works just in some containers, but not all.\nIt seems depending on distribution(Debian/Alpine/...) and language(Java/Node.js/Golang). The rule cannot be determined in actual behaviors.\nTherefore, all of the methods below are involved for safety when using self-signed certificates.\n\n- The host deploys /etc/ssl/certs/ca-certificates.crts to containers by volume mount.\n- Define env vars for self-signed certificates, such as GOINSECURE, NODE_TLS_REJECT_UNAUTHORIZED for each language.\n\n\n[back to top](#top)\n## \u003ca id=\"appendix\"/\u003eAppendix\n\n### \u003ca id=\"screenshots\"/\u003eScreen shots:\n\n| Create account | Sign-in|\n|:---|:---|\n|\u003cimg src=\"./docs/screenshots/1-bluesky-create-account.png\" style=\"height:45%; width:45%\"\u003e|\u003cimg src=\"./docs/screenshots/1-bluesky-sign-in.png\"  style=\"height:45%; width:45%\"\u003e|\n|\u003cimg src=\"./docs/screenshots/2-bluesky-choose-server.png\"  style=\"height:45%; width:45%\"\u003e|\u003cimg src=\"./docs/screenshots/2-bluesky-choose-server.png\"  style=\"height:45%; width:45%\"\u003e|\n|\u003cimg src=\"./docs/screenshots/3-bluesky-create-account.png\"  style=\"height:45%; width:45%\"\u003e|\u003cimg src=\"./docs/screenshots/3-bluesky-sign-in.png\"  style=\"height:45%; width:45%\"\u003e|\n\n### \u003ca id=\"sources\"/\u003eSources in use:\n\n| components     | url (origin)                                           |\n|----------------|:-------------------------------------------------------|\n| atproto        | https://github.com/bluesky-social/atproto.git          |\n| indigo         | https://github.com/bluesky-social/indigo.git           |\n| social-app     | https://github.com/bluesky-social/social-app.git       |\n| feed-generator | https://github.com/bluesky-social/feed-generator.git   |\n| pds            | https://github.com/bluesky-social/pds.git              |\n| ozone          | https://github.com/bluesky-social/ozone.git            |\n| did-method-plc | https://github.com/did-method-plc/did-method-plc.git   |\n| jetstream      | https://github.com/bluesky-social/jetstream.git        |\n\nother dependencies:\n\n| components     | url (origin)                                                            |\n|----------------|:------------------------------------------------------------------------|\n| reverse proxy  | https://github.com/caddyserver/caddy (official docker image of caddy:2) |\n| DNS server     | bind9 or others, such as https://github.com/itaru2622/docker-bind9.git  |\n\n[back to top](#top)\n\n### \u003ca id=\"sample-dns-config\"/\u003eDNS server configuration sample (bind9)\n\nDescription of test network:\n\n```\nDOMAIN for self-hosting: mysky.local.com\n\nIP:\n  - docker host for selfhost: 192.168.1.51\n  - DNS server:               192.168.1.27\n  - DNS forwarders:           8.8.8.8 (upper level DNS server;dns.google.)\n\nDNS A-Records:\n  -   mysky.local.com  : 192.168.1.51\n  - *.mysky.local.com  : 192.168.1.51\n```\n\nThe above would be described in bind9 configuration file as below:\n\n```\n::::::::::::::\n/etc/bind/named.conf\n::::::::::::::\ninclude \"/etc/bind/rndc.key\";\ncontrols {\n        inet 127.0.0.1 allow { 127.0.0.1; } keys { \"rndc-key\"; };\n};\noptions {\n        directory         \"/etc/bind\";\n        // UDP 53, from any\n        listen-on         { any; };\n        // HTTP 80, from any\n        listen-on  port 80  tls none http default  { any; };\n        listen-on-v6      { none; };\n        forwarders        { 8.8.8.8 ; };  # dns.gogle.\n        allow-recursion   { any; };\n        allow-query       { any; };\n        allow-query-cache { any; };\n        allow-transfer    { any; };\n};\nzone \"local.com\" { type master; file \"zone-local.com\"; allow-query { 0.0.0.0/0; }; allow-update { 0.0.0.0/0; }; allow-transfer { 0.0.0.0/0; }; };\n::::::::::::::\n/etc/bind/zone-local.com\n::::::::::::::\n$ORIGIN .\n$TTL 259200\t; 3 days\nlocal.com\t\tIN SOA\tlocal.com. root.local.com. (\n\t\t\t\t2024022809 ; serial\n\t\t\t\t3600       ; refresh (1 hour)\n\t\t\t\t900        ; retry (15 minutes)\n\t\t\t\t86400      ; expire (1 day)\n\t\t\t\t3600       ; minimum (1 hour)\n\t\t\t\t)\n\t\t\tNS\tlocal.com.\n\t\t\tA\t192.168.1.27\n$ORIGIN local.com.\n$TTL 3600\t; 1 hour\nmysky\t\tA\t192.168.1.51\n$ORIGIN mysky.local.com.\n*\t\t\tA\t192.168.1.51\n```\n\ncf. The simplest way to use the above DNS server(192.168.1.27) temporaly is\u003cbr\u003e\nto add it to /etc/resolv.conf as shown below on all testing machines\n(docker host, client machines for browsers)\n\n```\nnameserver 192.168.1.27\n```\n\n[back to top](#top)\n## \u003ca id=\"refs\"/\u003eReferences\n\nspecial thanks to prior works on self-hosting.\n   - https://github.com/ikuradon/atproto-starter-kit/tree/main\n   - https://github.com/bluesky-social/atproto/discussions/2026 and https://syui.ai/blog/post/2024/01/08/bluesky/\n\nhacks in bluesky:\n   - https://github.com/bluesky-social/social-app/blob/main/docs/build.md\n   - https://github.com/bluesky-social/indigo/blob/main/HACKING.md\n   - https://github.com/bluesky-social/ozone/blob/main/HOSTING.md\n   - https://github.com/bluesky-social/pds/blob/main/installer.sh\n\n[back to top](#top)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fitaru2622%2Fbluesky-selfhost-env","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fitaru2622%2Fbluesky-selfhost-env","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fitaru2622%2Fbluesky-selfhost-env/lists"}