{"id":15152956,"url":"https://github.com/adamcanray/private-ipfs-cluster-data-replication","last_synced_at":"2026-02-06T22:01:41.523Z","repository":{"id":217656245,"uuid":"744007877","full_name":"adamcanray/Private-IPFS-Cluster-Data-Replication","owner":"adamcanray","description":"Private IPFS Cluster for Data Replication","archived":false,"fork":false,"pushed_at":"2024-03-14T04:38:20.000Z","size":12337,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-13T14:19:15.581Z","etag":null,"topics":["data-replication","docker","ipfs"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/adamcanray.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-16T12:58:43.000Z","updated_at":"2024-01-17T11:47:37.000Z","dependencies_parsed_at":"2024-03-14T05:30:29.563Z","dependency_job_id":null,"html_url":"https://github.com/adamcanray/Private-IPFS-Cluster-Data-Replication","commit_stats":{"total_commits":20,"total_committers":1,"mean_commits":20.0,"dds":0.0,"last_synced_commit":"8286a9f40d7e4d0184f8f823c779117e4ce09c6b"},"previous_names":["adamcanray/private-ipfs-cluster-data-replication"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adamcanray%2FPrivate-IPFS-Cluster-Data-Replication","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adamcanray%2FPrivate-IPFS-Cluster-Data-Replication/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adamcanray%2FPrivate-IPFS-Cluster-Data-Replication/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adamcanray%2FPrivate-IPFS-Cluster-Data-Replication/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/adamcanray","download_url":"https://codeload.github.com/adamcanray/Private-IPFS-Cluster-Data-Replication/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247645400,"owners_count":20972471,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-replication","docker","ipfs"],"created_at":"2024-09-26T16:42:14.956Z","updated_at":"2026-02-06T22:01:41.495Z","avatar_url":"https://github.com/adamcanray.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Private IPFS Cluster for Data Replication\n\nThis repository provides a comprehensive guide and setup for building a private IPFS (InterPlanetary File System) cluster designed for secure and efficient data replication. Inspired by the insightful work at [Eleks Research](https://eleks.com/research/ipfs-network-data-replication/), this repository covers the creation of a IPFS Swarm network, joining nodes to the swarm, configuring IPFS, and establishing a robust private network for enhanced data replication and distribution.\n\n## Key Features:\n\n- **IPFS Swarm Integration**: Initiate and manage a IPFS Swarm network to orchestrate IPFS nodes for improved collaboration and resource utilization.\n\n- **Cross-Platform Compatibility**: Utilize Alpine Linux and ARM64 architecture for a lightweight and efficient containerized environment.\n\n- **IPFS Configuration**: Configure IPFS nodes for private access, adjusting addresses, and generating the necessary swarm key for secure communication.\n\n- **Bootstrapping IPFS Nodes**: Learn the essentials of bootstrapping IPFS nodes, ensuring efficient communication and replication within the private cluster.\n\n- **Service Management**: Implement service management with OpenRC in Alpine Linux, converting from a systemd service to an OpenRC service.\n\n- **Testing and Usage**: Verify the functionality of your private IPFS network by testing access to Content Identifiers (CIDs) and ensuring smooth data replication.\n\n## Run:\n\n1.  **Prepare common files**:\n\n    ```bash\n    chmod +x ./prepare.sh\n    ./prepare.sh\n    ```\n\n    This will create the necessary directories and files for the manager/worker docker image.\n\n2.  **Setup Swarm IPFS**:\n\n    - Follow the instructions in [run-guide](./RUN-GUIDE.md) for initializing the IPFS Swarm network.\n\n## Usage\n\n1.  **Check**:\n\n    We can go open docker container terminal with `docker exec -it \u003ccontainer\u003e /bin/sh` command, then we can test the IPFS with the following commands:\n\n    Make sure all necessary files are in place:\n\n    ```bash\n      ls -a\n      ls /vol/swarm -a\n    ```\n\n    Check the IPFS configuration:\n\n    ```bash\n    cat ~/.ipfs/swarm.key\n    cat ~/.ipfs/config\n    ```\n\n    Check the IPFS Daemon service status:\n\n    ```bash\n    cat etc/init.d/ipfs\n    rc-status -a\n    rc-service ipfs status\n    ```\n\n    Check swarm network:\n\n    ```bash\n    ipfs bootstrap\n    ipfs swarm peers\n    ```\n\n    \u003e Make sure if the IPFS Daemon is running. If not, we can start it manually for each node with `rc-service ipfs start` command.\n\n2.  **Testing**:\n\n    You can test the IPFS with the following commands:\n\n    ```bash\n    ipfs add -r /vol/swarm\n    ipfs pin ls -t recursive\n    ipfs pin rm -r \u003ccid\u003e\n    ipfs repo gc\n    ```\n\n    Test the IPFS Gateway with the following command:\n\n    ```bash\n    curl http://\u003cnode_ip_address\u003e:8080/ipfs/\u003ccid\u003e\n    ```\n\n    \u003c!-- image preview-1.1.png --\u003e\n\n    ![image1.1](preview-1.1.png)\n    ![image1.2](preview-1.2.png)\n\n    Also we can test upload/add file via api server with the following:\n\n    ```bash\n    curl -X POST -F file=@/path/to/your/file http://\u003cnode_ip_address\u003e:5001/api/v0/add\n    ```\n\n    \u003e Since we [doesn't expose the API server to the public](https://github.com/adamcanray/Private-IPFS-Cluster-Data-Replication/blob/d547b12eafdca7d72583f1123436ffde9a354b90/common/ipfs-update-config.sh#L15), we can use other alternative to upload file to the IPFS network, more on [rpc docs](https://docs.ipfs.tech/reference/kubo/rpc/)\n\n## Without Docker Compose\n\nYou can see [Private-IPFS-Manager](https://github.com/adamcanray/Private-IPFS-Manager) and [Private-IPFS-Worker](https://github.com/adamcanray/Private-IPFS-Worker) repository to run the manager and worker node without docker compose.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadamcanray%2Fprivate-ipfs-cluster-data-replication","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fadamcanray%2Fprivate-ipfs-cluster-data-replication","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadamcanray%2Fprivate-ipfs-cluster-data-replication/lists"}