{"id":19747482,"url":"https://github.com/moosefs/moosefs-docker-cluster","last_synced_at":"2025-04-30T08:32:31.099Z","repository":{"id":49626197,"uuid":"93032574","full_name":"moosefs/moosefs-docker-cluster","owner":"moosefs","description":"Multiple node MooseFS cluster on Docker","archived":false,"fork":false,"pushed_at":"2023-07-17T12:39:36.000Z","size":165,"stargazers_count":73,"open_issues_count":2,"forks_count":25,"subscribers_count":10,"default_branch":"master","last_synced_at":"2023-08-06T08:03:24.252Z","etag":null,"topics":["cluster","containers","docker","filesystem","moosefs","moosefs-cluster","storage","tests"],"latest_commit_sha":null,"homepage":"https://moosefs.com","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/moosefs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-06-01T08:05:30.000Z","updated_at":"2023-03-25T21:31:29.000Z","dependencies_parsed_at":"2023-02-18T10:45:18.511Z","dependency_job_id":null,"html_url":"https://github.com/moosefs/moosefs-docker-cluster","commit_stats":null,"previous_names":[],"tags_count":3,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moosefs%2Fmoosefs-docker-cluster","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moosefs%2Fmoosefs-docker-cluster/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moosefs%2Fmoosefs-docker-cluster/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moosefs%2Fmoosefs-docker-cluster/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/moosefs","download_url":"https://codeload.github.com/moosefs/moosefs-docker-cluster/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224202883,"owners_count":17272838,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cluster","containers","docker","filesystem","moosefs","moosefs-cluster","storage","tests"],"created_at":"2024-11-12T02:17:58.636Z","updated_at":"2024-11-12T02:17:59.079Z","avatar_url":"https://github.com/moosefs.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MooseFS Docker Cluster\n\nThis is a basic configuration of a multiple-node MooseFS cluster based on the Debian Buster Docker image. It consists of a Master Server, a CGI Monitoring Interface Server, 4 Chunkservers, and one Client. After a successful installation, you will have a fully working MooseFS cluster to play with its amazing features.\n\n# Updates\n\n- All MooseFS processes are now correctly handling signals.\n- Metadata and data are now persistent and mounted as volumes.\n- TEST and PROD moosefs master metadata behavior defined by `MFS_ENV` variable.\n- Specify storage size per chunk server (env: `SIZE`) default not defined. Depends on your local storage free space.\n- Specify label per chunk server (env: `LABEL`, default: *empty*).\n- Switched to *debian:buster* as a base image.\n- Example with 4 chunk servers (labels: M, MB, MB, B).\n\n# Cluster configurations\n\n**File docker-compose.yml**\n\n- Master Server: `172.20.0.2`\n- CGI Monitoring Interface: [http://localhost:9425](http://localhost:9425), on Linux also [http://172.20.0.3:9425](http://172.20.0.3:9425)\n- Metalogger: `172.20.0.4`\n- Chunkserver 1: `172.20.0.11`, labels: `M`\n- Chunkserver 2: `172.20.0.12`, labels: `M, B`\n- Chunkserver 3: `172.20.0.13`, labels: `M, B`\n- Chunkserver 4: `172.20.0.14`, labels: `B`\n- Client: `172.168.20.0.100`\n\n# Setup\n\nInstall Docker with Docker Composer from [https://docs.docker.com/compose/install](https://docs.docker.com/compose/install)\n\nClone MooseFS docker config files:\n\n```\ngit clone https://github.com/moosefs/moosefs-docker-cluster\ncd moosefs-docker-cluster\n```\n\nBuild and run:\n\n```\ndocker-compose build\ndocker-compose up\n```\n\nOn Linux OS run `docker-compose` as root:\n\n```\nsudo -E docker-compose build\nsudo -E docker-compose up\n```\n\nYou can also run `docker-compose` in detached mode. All running Docker nodes will run in the background, so Docker console output will be invisible.\n\n```\nsudo -E docker-compose build\nsudo -E docker-compose up -d\n```\n\nYou can check if instances are running:\n\n```\ndocker ps\n```\n\nYou should have 1 Master Server, 1 Metalogger, 4 Chunkservers and 1 Client running (first configuration). The expected output should be similar to the following:\n\n```\nCONTAINER ID        IMAGE                                  COMMAND                  CREATED             STATUS              PORTS                    NAMES\nabf5910b53bc        moosefsdockercluster_mfsclient         \"mfsmount -f /mnt/mo…\"   7 minutes ago       Up 7 minutes                                 mfsclient\n7a1152cc31f3        moosefsdockercluster_mfschunkserver3   \"chunkserver.sh\"         7 minutes ago       Up 7 minutes        9422/tcp                 mfschunkserver3\nb8c2cd770187        moosefsdockercluster_mfschunkserver2   \"chunkserver.sh\"         7 minutes ago       Up 7 minutes        9422/tcp                 mfschunkserver2\n100f20683b3a        moosefsdockercluster_mfschunkserver1   \"chunkserver.sh\"         7 minutes ago       Up 7 minutes        9422/tcp                 mfschunkserver1\n68ffb70ab361        moosefsdockercluster_mfschunkserver4   \"chunkserver.sh\"         7 minutes ago       Up 7 minutes        9422/tcp                 mfschunkserver4\n82a2c3bd831d        moosefsdockercluster_mfsmetalogger     \"metalogger.sh\"          7 minutes ago       Up 7 minutes                                 mfsmetalogger\n05736e4bdd3c        moosefsdockercluster_mfscgi            \"mfscgiserv -f\"          7 minutes ago       Up 7 minutes        0.0.0.0:9425-\u003e9425/tcp   mfscgi\ne83a1fb062a1        moosefsdockercluster_mfsmaster         \"master.sh\"              7 minutes ago       Up 7 minutes        9419-9421/tcp            mfsmaster\n```\n\n# Attach / detach to / from container\n\nFor example, if you like to **attach** to the client node execute this command:\n\n```\ndocker exec -it mfsclient bash\n```\n\nTo **detach** from container, just press `Ctrl + D` keys combination.\n\n# MooseFS Client\n\nMooseFS filesystem is mounted at `/mnt/moosefs`. If everything is ok you should see this ASCII art:\n\n```\ncat /mnt/moosefs/.mooseart\n \\_\\            /_/\n    \\_\\_    _/_/\n        \\--/\n        /OO\\_--____\n       (__)        )\n        ``\\    __  |\n           ||-'  `||\n           ||     ||\n           \"\"     \"\"\n```\n\n# MooseFS CGI Monitoring Interface\n\nThe MooseFS CGI Monitoring Interface is available here: [http://localhost:9425](http://localhost:9425).\n\nAlso on Linux, CGI Server container is available at the IP address: [http://172.20.0.3:9425](http://172.20.0.3:9425) (be aware of a local `172.20.0.x` network).\n\n![MooseFS CGI](https://github.com/moosefs/moosefs-docker-cluster/raw/master/images/cgi.png)\n\n# Persistence\n\nYour MooseFS Docker cluster is persistent. It means all files you created in the `/mnt/moosefs` folder will remain there even after turning containers off.\nAll data and metadata files are stored in the host `./data` directory.\n\n# Pass config as env variable\n\nThere might be situations where you would want to setup a config file on the container start.\nFor that scenario you can pass the config file as a base64 encoded text. For example lets say\nyou want to setup your chunk servers to connect to master in k8s cluster where IP's are dynamically assigned\nto pods. You have you master yaml definition set up as:\n```\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n  name: moosefs-master\n  namespace: storage\nspec:\n  replicas: 1\n  selector:\n    matchLabels:\n      app: moosefs-master\n  template:\n    metadata:\n      labels:\n        app: moosefs-master\n    spec:\n      nodeSelector:\n        \"beta.kubernetes.io/os\": linux\n      containers:\n      - name: moosefs-master\n        image: moosefs/master:latest\n        resources:\n          requests:\n            cpu: 100m\n            memory: 128Mi\n          limits:\n            cpu: 250m\n            memory: 256Mi\n        ports:\n        - containerPort: 9419\n        - containerPort: 9420\n        - containerPort: 9421\n        volumeMounts:\n        - name: moosefs-master-mfs\n          mountPath: /var/lib/mfs\n      volumes:\n      - name: moosefs-master-mfs\n        azureDisk:\n          kind: Managed\n          diskName: MooseMasterMfs\n          diskURI: /subscriptions/\u003csubscriptionID\u003e/resourceGroups/\u003cresourceGroup\u003e/providers/Microsoft.Compute/disks/MooseMasterMfs\n---\napiVersion: v1\nkind: Service\nmetadata:\n  name: moosefs-master\n  namespace: storage\nspec:\n  type: NodePort\n  ports:\n  - port: 9419\n    targetPort: 9419\n    name: listen-metalogger\n  - port: 9420\n    targetPort: 9420\n    name: listen\n  - port: 9421\n    targetPort: 9421\n    name: listen-client\n  selector:\n    app: moosefs-master\n```\n\nThis will reserve an IP in the cluster where the ports will be reached.\n\nIn order for your chunkservers to automatically connect to this IP you would need to have your `mfschunkserver.cfg` defined as:\n```\nMASTER_HOST = $MOOSEFS_MASTER_SERVICE_HOST\nCSSERV_LISTEN_PORT = $MOOSEFS_CHUNKSERVER_SERVICE_PORT\nDATA_PATH = /mnt/hdd0/mfs\n```\n`MOOSEFS_MASTER_SERVICE_HOST` variable is set by k8s cluster and contains IP where the master service is accessible by. `MOOSEFS_CHUNKSERVER_SERVICE_PORT` this is the port on which we will expose our chunk server.\n\nBase64 encoded config data is:\n```\nTUFTVEVSX0hPU1QgPSAkTU9PU0VGU19NQVNURVJfU0VSVklDRV9IT1NUCkNTU0VSVl9MSVNURU5fUE9SVCA9ICRNT09TRUZTX0NIVU5LU0VSVkVSX1NFUlZJQ0VfUE9SVApEQVRBX1BBVEggPSAvbW50L2hkZDAvbWZzCg==\n```\n\nNow we can spin up chunkserver(s) with:\n```\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n  name: moosefs-chunkserver-1\n  namespace: storage\nspec:\n  replicas: 1\n  selector:\n    matchLabels:\n      app: moosefs-chunkserver-1\n  template:\n    metadata:\n      labels:\n        app: moosefs-chunkserver-1\n    spec:\n      nodeSelector:\n        \"beta.kubernetes.io/os\": linux\n      containers:\n      - name: moosefs-chunkserver-1\n        env:\n        - name: MFS_CHUNKSERVER_CONFIG\n          value: TUFTVEVSX0hPU1QgPSAkTU9PU0VGU19NQVNURVJfU0VSVklDRV9IT1NUCkNTU0VSVl9MSVNURU5fUE9SVCA9ICRNT09TRUZTX0NIVU5LU0VSVkVSX1NFUlZJQ0VfUE9SVApEQVRBX1BBVEggPSAvbW50L2hkZDAvbWZzCg==\n        - name: SIZE\n          value: 16\n        image: moosefs/chunkserver:latest\n        ports:\n        - containerPort: 9422\n        volumeMounts:\n        - name: moosefs-chunkserver-data-1\n          mountPath: /mnt/hdd0\n      volumes:\n      - name: moosefs-chunkserver-data-1\n        azureDisk:\n          kind: Managed\n          diskName: MfsHangfireData1\n          diskURI: /subscriptions/\u003csubscriptionID\u003e/resourceGroups/\u003cresourceGroup\u003e/providers/Microsoft.Compute/disks/MfsHangfireData1\n---\napiVersion: v1\nkind: Service\nmetadata:\n  name: moosefs-chunkserver-1\n  namespace: storage\nspec:\n  type: NodePort\n  ports:\n  - port: 9422\n    targetPort: 9422\n  selector:\n    app: moosefs-chunkserver-1\n```\nRepeat this for other chunk servers modifying your base64 string accordingly. If you leave all the chunkservers on default port `9422` you can use same base64 encoded string `TUFTVEVSX0hPU1QgPSAkTU9PU0VGU19NQVNURVJfU0VSVklDRV9IT1NUCkRBVEFfUEFUSCA9IC9tbnQvaGRkMC9tZnMK` which will only set correct `MASTER_HOST` and `DATA_PATH`\n\n# Docker Hub\n\n| Image name | Image size | Pulls | Stars | Build |\n|:-----|:-----|:-----|:-----|:-----|\n| [moosefs/master](https://hub.docker.com/r/moosefs/master/) | ![Docker Image Size (latest by date)](https://img.shields.io/docker/image-size/moosefs/master?sort=date) | [![master](https://img.shields.io/docker/pulls/moosefs/master)](https://hub.docker.com/r/moosefs/master/) | ![master](https://img.shields.io/docker/stars/moosefs/master) | ![master](https://img.shields.io/docker/cloud/build/moosefs/master) |\n| [moosefs/chunkserver](https://hub.docker.com/r/moosefs/chunkserver/) | ![Docker Image Size (latest by date)](https://img.shields.io/docker/image-size/moosefs/chunkserver?sort=date) | [![chunkserver](https://img.shields.io/docker/pulls/moosefs/chunkserver)](https://hub.docker.com/r/moosefs/chunkserver/) | ![chunkserver](https://img.shields.io/docker/stars/moosefs/chunkserver) | ![chunkserver](https://img.shields.io/docker/cloud/build/moosefs/chunkserver) |\n| [moosefs/client](https://hub.docker.com/r/moosefs/client/) | ![Docker Image Size (latest by date)](https://img.shields.io/docker/image-size/moosefs/client?sort=date) | [![client](https://img.shields.io/docker/pulls/moosefs/client)](https://hub.docker.com/r/moosefs/client/) | ![client](https://img.shields.io/docker/stars/moosefs/client) | ![client](https://img.shields.io/docker/cloud/build/moosefs/client) |\n| [moosefs/metalogger](https://hub.docker.com/r/moosefs/metalogger/) | ![Docker Image Size (latest by date)](https://img.shields.io/docker/image-size/moosefs/metalogger?sort=date) | [![metalogger](https://img.shields.io/docker/pulls/moosefs/metalogger)](https://hub.docker.com/r/moosefs/cgi/)    | ![metalogger](https://img.shields.io/docker/stars/moosefs/metalogger)  | ![metalogger](https://img.shields.io/docker/cloud/build/moosefs/metalogger) |\n| [moosefs/cgi](https://hub.docker.com/r/moosefs/cgi/) | ![Docker Image Size (latest by date)](https://img.shields.io/docker/image-size/moosefs/cgi?sort=date) | [![cgi](https://img.shields.io/docker/pulls/moosefs/cgi)](https://hub.docker.com/r/moosefs/cgi/) | ![cgi](https://img.shields.io/docker/stars/moosefs/cgi)  | ![cgi](https://img.shields.io/docker/cloud/build/moosefs/cgi) |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmoosefs%2Fmoosefs-docker-cluster","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmoosefs%2Fmoosefs-docker-cluster","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmoosefs%2Fmoosefs-docker-cluster/lists"}