{"id":13507746,"url":"https://github.com/tomaae/storj-deployment","last_synced_at":"2025-03-30T09:33:01.875Z","repository":{"id":100238486,"uuid":"572342231","full_name":"tomaae/storj-deployment","owner":"tomaae","description":"Full compose multinode deployment for storj-node with exporters and grafana","archived":true,"fork":false,"pushed_at":"2022-12-09T01:30:32.000Z","size":133,"stargazers_count":12,"open_issues_count":1,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-29T12:55:17.138Z","etag":null,"topics":["docker-compose","storj","storj-exporter","storj-node"],"latest_commit_sha":null,"homepage":"","language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tomaae.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-11-30T04:00:08.000Z","updated_at":"2025-02-07T14:23:54.000Z","dependencies_parsed_at":"2023-05-13T01:00:36.211Z","dependency_job_id":null,"html_url":"https://github.com/tomaae/storj-deployment","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tomaae%2Fstorj-deployment","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tomaae%2Fstorj-deployment/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tomaae%2Fstorj-deployment/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tomaae%2Fstorj-deployment/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tomaae","download_url":"https://codeload.github.com/tomaae/storj-deployment/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246301963,"owners_count":20755512,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker-compose","storj","storj-exporter","storj-node"],"created_at":"2024-08-01T02:00:38.525Z","updated_at":"2025-03-30T09:33:01.869Z","avatar_url":"https://github.com/tomaae.png","language":"PHP","funding_links":[],"categories":["Clients with Storj support and integrations"],"sub_categories":[],"readme":"# storj-deployment\nFull compose multinode deployment for storj-node with exporters and grafana.\n\nThis implementation aims to gather all available information in single place and make deployment as simple as possible.\n\n## Deployment example\nGrafana storj\n\n![UI](https://user-images.githubusercontent.com/23486452/206063260-b472fdf0-bc39-4ed4-a1b2-dafa934a8733.png)\n\nGrafana host\n\n![UI](https://user-images.githubusercontent.com/23486452/206063367-0df86143-a420-45a5-946f-c77544d51c57.png)\n\nList of web interfaces\n\n![UI List](https://user-images.githubusercontent.com/23486452/205275296-3524e3db-f525-4d6b-818e-08e2646a51fe.png)\n\nstorj-node stack\n\n![storj-node stack](https://user-images.githubusercontent.com/23486452/205275465-39f29074-6b33-412d-b4e0-8c9678e09f12.png)\n\nstorj-ui stack\n\n![storj-ui stack](https://user-images.githubusercontent.com/23486452/205275530-d2e18160-2909-4f4f-9eb7-ea859280a95a.png)\n\nPortainer\n\n![Docker](https://user-images.githubusercontent.com/23486452/205275612-7faa28e9-57e2-4b1e-a0b9-7d2a2d3fc65a.png)\n\n*Portainer is not included in this deployment guide*\n\n\n## Per node compose deployment\n* storj-node with log to file\n* preconfigured zksync\n* logrotate for storj-node\n* storj exporter\n* storj log exporter\n* storj updater\n* running on shared dedicated internal network\n\n## Other\n* docker host OS monitoring\n* prometheus + grafana configuration\n* grafana storj and host os monitoring graphs\n* simple website for host displaying all available storj web interfaces\n* storj multinode optional deployment\n\n## Tutorial\n[![Tutorial](https://user-images.githubusercontent.com/23486452/206584124-81c8a3c5-0302-49bd-a1e9-5eab22f88808.png)](https://www.youtube.com/watch?v=7C2mGGZkcMk)\n\n## Credits\nImplementation includes following:\n* https://github.com/storj/storj\n* https://github.com/anclrii/Storj-Exporter\n* https://github.com/anclrii/Storj-Exporter-dashboard\n* https://github.com/kevinkk525/storj-log-exporter\n* https://github.com/prometheus/node_exporter\n* https://github.com/rfmoz/grafana-dashboards\n\n\n# Prepare for deployment\nThis is one time step to build a docker image we will use in node deployment.\n\n## Build storj-log-exporter docker image\n```\ncd ~\ngit clone https://github.com/kevinkk525/storj-log-exporter\ncd storj-log-exporter\nsudo docker build -t storj-log-exporter .\n```\n\n# Node deployment\nExample for node \"01\". Increment number as needed.\n\n## Prepare storj node implementation\nGet your auth token and Generate node identity as normal with name nodeXX, replace XX with node number. Example: node01\nhttps://www.storj.io/host-a-node\n\nLinux example for node01:\n```\ncd ~\nwget https://github.com/storj/storj/releases/latest/download/identity_linux_amd64.zip\nunzip -o identity_linux_amd64.zip\nchmod +x identity\n./identity create node01\n./identity authorize node01 \u003cemail:characterstring\u003e\ngrep -c BEGIN ~/.local/share/storj/identity/node01/ca.cert\ngrep -c BEGIN ~/.local/share/storj/identity/node01/identity.cert\n```\n*grep should result in 2 and 3 respectively.*\n\nPort forward 140XX, replace XX with node number. Example: 14001\n\n## Initialize storj node\n```\ndocker run --rm -e SETUP=\"true\" \\\n    --user $(id -u):$(id -g) \\\n    --mount type=bind,source=\"/root/.local/share/storj/identity/node01\",destination=/app/identity \\\n    --mount type=bind,source=\"/mnt/node01\",destination=/app/config \\\n    --name storj-node01 storjlabs/storagenode:latest\n```\n\n## Deploy storj node\nCheckout storj-node stack for new node\n```\ncd ~\nrm /mnt/node01/config.yaml\nsvn checkout https://github.com/tomaae/storj-deployment.git/trunk/storj-nodeXX storj-node01\n```\n\nEdit stack configuration:\n```\nsed -i 's/XX/01/g' ./storj-node01/.env\n```\n\nEdit storj-node01/.env attributes as needed:\n```\nNODE_ID=\u003c2 number node id\u003e\nNODE_EMAIL=\u003ccontact email\u003e\nNODE_EXTERNAL=\u003cexternal fqdn\u003e:150XX\nNODE_STORAGE=\u003cstorj node size\u003e\nNODE_WALLET=\u003cwallet address\u003e\nNODE_MNT=\u003cstorj node config mount point\u003e\nNODE_IDENTITY=\u003cstorj node iudentity directory\u003e\n```\n*node ID, ports and NODE_IDENTITY should be preconfigured correctly*\n\nDeploy node\n```\ncd storj-node01\ndocker compose up -d\n```\n\n# Grafana deployment\nOne time deployment\n\nCheckout storj-ui stack\n```\ncd ~\nsvn checkout https://github.com/tomaae/storj-deployment.git/trunk/storj-ui\n```\n\nEdit stack configuration storj-ui/.env\n```\nPROMETHEUS_MOUNT=\u003cui mount pointmount point\u003e/prometheus\nGRAFANA_MOUNT=\u003cui mount point\u003e/grafana\n\nGF_SECURITY_ADMIN_PASSWORD=\u003cpassword\u003e\nGF_USERS_ALLOW_SIGN_UP=false\nGF_INSTALL_PLUGINS=yesoreyeram-boomtable-panel\n\nDOCKER_HOST_HOSTNAME=\u003cdocker host fqdn\u003e\n```\n\nEdit storj-ui/prometheus.yml, append block for each node, replace XX with node ID\n```\n  - job_name: storj-nodeXX\n    scrape_interval: 60s\n    scrape_timeout: 20s\n    metrics_path: /\n    static_configs:\n      - targets: [\"storj-exporterXX:9651\", \"storj-log-exporterXX:9144\"]\n        labels:\n          instance: \"nodeXX\"\n```\n\nDeploy storj-ui\n```\ncd storj-ui\ndocker compose up -d\n```\n\nImport grafana dashboard:\n\nhttps://raw.githubusercontent.com/tomaae/storj-deployment/main/storj-ui/dashboard_exporter_combined.json\n\n# (Optional) Deploy docker host statistics\nIf you want docker host statistics, deploy for each docker host\n\nCheckout storj-host stack\n```\ncd ~\nmkdir -p /mnt/exporter/prometheus /mnt/exporter/grafana\nchown 777 /mnt/exporter/prometheus /mnt/exporter/grafana\nsvn checkout https://github.com/tomaae/storj-deployment.git/trunk/storj-hostXX storj-host01\nsed -i 's/XX/01/g' ./storj-host01/.env\ncd storj-host01\ndocker compose up -d\n```\n\nEdit storj-ui/prometheus.yml, append block for each host, replace XX with host ID\n```\n  - job_name: storj-hostXX\n    static_configs:\n    - targets: ['host-exporterXX:9100']\n      labels:\n        instance: \"hostXX\"\n```\n\nDeploy updated storj-ui\n```\ncd storj-ui\ndocker compose down\ndocker compose up -d\n```\n\nImport grafana dashboard:\n\nhttps://raw.githubusercontent.com/tomaae/storj-deployment/main/storj-ui/node-exporter-full.json\n\n\n# (Optional) multinode deployment\nOne time deployment if you want to also deploy official multinode\n\nGenerate identity for multinode:\n```\ncd ~\n./identity create multinode --difficulty 10\n```\n\nDeploy docker image:\n```\ndocker run -d --restart unless-stopped \\\n    --user $(id -u):$(id -g) \\\n    -p 10000:15002/tcp \\\n    --network storj-bridge\n    --mount type=bind,source=\"/root/.local/share/storj/identity/multinode/ca.key\",destination=/app/identity \\\n    --mount type=bind,source=\"/mnt/multinode\",destination=/app/config \\\n    --name storj-multinode storjlabs/multinode:latest\n```\n\nIssue API key:\n```\ndocker exec -it storj-node01 ./storagenode issue-apikey --identity-dir /app/identity --config-dir /app/config --log.output stderr\n```\n*Change node01 to appropriate node number*\n\nmultinode UI will be available on port 10000\nWhen importing new nodes, use internal network addresses instead of public:\n```\nPublic IP Address:\nstorj-node01:28967\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftomaae%2Fstorj-deployment","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftomaae%2Fstorj-deployment","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftomaae%2Fstorj-deployment/lists"}