{"id":13439717,"url":"https://github.com/moosefs/moosefs","last_synced_at":"2025-05-14T09:09:05.947Z","repository":{"id":37412513,"uuid":"50505775","full_name":"moosefs/moosefs","owner":"moosefs","description":"MooseFS Distributed Storage – Open Source, Petabyte, Fault-Tolerant, Highly Performing, Scalable Network Distributed File System / Software-Defined Storage","archived":false,"fork":false,"pushed_at":"2025-03-31T11:37:29.000Z","size":6908,"stargazers_count":1768,"open_issues_count":182,"forks_count":218,"subscribers_count":109,"default_branch":"master","last_synced_at":"2025-04-03T19:58:16.879Z","etag":null,"topics":["backup","block-storage","distributed-computing","distributed-file-storage","distributed-file-system","distributed-storage","ditributed-systems","erasure-coding","file-storage","file-system","filesystem","fuse","high-availability","media","posix","posix-compliant","snapshots","software-defined-storage","storage","tiering"],"latest_commit_sha":null,"homepage":"https://moosefs.com","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/moosefs.png","metadata":{"files":{"readme":"README","changelog":"NEWS","contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-01-27T12:28:13.000Z","updated_at":"2025-04-02T21:49:05.000Z","dependencies_parsed_at":"2023-10-11T16:24:37.385Z","dependency_job_id":"26f48668-56b3-473b-8bf6-bd4fce061390","html_url":"https://github.com/moosefs/moosefs","commit_stats":{"total_commits":636,"total_committers":15,"mean_commits":42.4,"dds":0.06289308176100628,"last_synced_commit":"5afd7b57c065d6b36ebe755ccffd17adf33b1975"},"previous_names":[],"tags_count":129,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moosefs%2Fmoosefs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moosefs%2Fmoosefs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moosefs%2Fmoosefs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moosefs%2Fmoosefs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/moosefs","download_url":"https://codeload.github.com/moosefs/moosefs/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248335017,"owners_count":21086492,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backup","block-storage","distributed-computing","distributed-file-storage","distributed-file-system","distributed-storage","ditributed-systems","erasure-coding","file-storage","file-system","filesystem","fuse","high-availability","media","posix","posix-compliant","snapshots","software-defined-storage","storage","tiering"],"created_at":"2024-07-31T03:01:16.485Z","updated_at":"2025-04-11T03:27:44.241Z","avatar_url":"https://github.com/moosefs.png","language":"C","funding_links":[],"categories":["C","Operation System","C (61)","Interaction","others","Distributed file systems"],"sub_categories":["Network Distributed File System","Network Distributed File Systems"],"readme":"MooseFS - A Petabyte Distributed File System\n\nMooseFS is a Petabyte Open Source Network Distributed File System. It is easy\nto deploy and maintain, highly reliable, fault tolerant, highly performing,\neasily scalable and POSIX compliant.\n\nMooseFS spreads data over a number of commodity servers, which are visible\nto the user as one resource. For standard file operations MooseFS acts\nlike ordinary Unix-like file system:\n\n  * A hierarchical structure – directory tree\n  * Stores POSIX file attributes – permissions, last access and modification\n    times, etc.\n  * Supports ACLs\n  * Supports POSIX and BSD file locks – including support for distributed\n    file locking\n  * Supports special files – block and character devices, pipes and sockets\n  * Supports symbolic links – file names pointing to target files,\n    not necessarily on MooseFS\n  * Supports hard links – different names of files which refer to the same\n    data on MooseFS\n\n\nDistinctive MooseFS features:\n=============================\n\n  * High reliability – files are stored in several copies on separate servers.\n    The number of copies is a configurable parameter, even per each file\n  * No Single Point of Failure – all hardware and software components may be\n    redundant\n  * Parallel data operations – many clients can access many files concurrently\n  * Capacity can be dynamically expanded by simply adding new servers/disks\n    on the fly\n  * Retired hardware may be removed on the fly\n  * Deleted files are retained for a configurable period of time (a file system\n    level \"trash bin\")\n  * Coherent, \"atomic\" snapshots of files, even while the files are being\n    written/accessed\n  * Access to the file system can be limited based on IP address and/or password\n    (similarly as in NFS)\n  * Data tiering – supports different storage policies for different\n    files/directories in Storage Classes mechanism\n  * Per-directory, \"project\" quotas – configurable per RAW space, usable space,\n    number of inodes with hard and soft quotas support\n  * Apart from file system storage, MooseFS also provides block storage (mfsbdev)\n  * Efficient, pure C implementation\n  * Ethernet support\n\n\nSupported platforms\n===================\n\nMooseFS can be installed on any POSIX compliant operating system including\nvarious Linux distributions, FreeBSD and macOS:\n\n  * Ubuntu\n  * Debian\n  * RHEL / CentOS\n  * OpenSUSE\n  * FreeBSD\n  * macOS\n\nMooseFS Client uses FUSE library, available for:\nLinux \u0026 BSD (https://github.com/libfuse/libfuse)\nand macOS (https://github.com/osxfuse/osxfuse).\n\nThere is a separate MooseFS Client for Microsoft Windows available,\nbuilt on top of Dokany (https://github.com/dokan-dev/dokany).\n\n\nGetting started\n===============\n\nYou can install MooseFS using your favorite package manager on one\nof the following platforms using officially supported repositories\n(https://moosefs.com/download):\n\n  * Ubuntu 16 / 18 / 20 / 22 / 24\n  * Debian 9 / 10 / 11 / 12 / 13\n  * RHEL / CentOS 7 / 8 / 9\n  * FreeBSD 11 / 12 / 13 / 14\n  * macOS 10.12+\n  * Ubuntu 20 / 22 – Raspberry Pi\n  * Debian 11 / 12 – Raspberry Pi\n\nPackages for CentOS 6 are also available, but no longer supported.\n\nDebian packages are compatible with Proxmox.\nCentOS packages are compatible with Rocky Linux, AlmaLinux and openSUSE Leap.\n\nMinimal set of packages, which are needed to run MooseFS:\n  * moosefs-master – MooseFS Master Server for metadata servers,\n  * moosefs-chunkserver – MooseFS Chunkserver for data storage servers,\n  * moosefs-client – MooseFS Client – client side package to mount\n    the filesystem.\n\n\nSource code\n-----------\n\nFeel free to download the source code from our GitHub code repository!\n\nInstall the following dependencies before building MooseFS from sources:\n  * Debian/Ubuntu: sudo apt install build-essential libpcap-dev zlib1g-dev libfuse3-dev pkg-config\n  (if you don't have FUSE v. 3 in your system, use: \n  sudo apt install build-essential libpcap-dev zlib1g-dev libfuse-dev pkg-config)\n  * CentOS/RHEL: sudo yum install gcc gcc-c++ make libpcap-devel zlib-devel fuse3-devel pkgconfig\n  (if you don't have FUSE v. 3 in your system, use:\n  sudo yum install gcc make libpcap-devel zlib-devel fuse-devel pkgconfig)\n\nRecommended packages:\n  * Debian/Ubuntu: sudo apt install fuse3\n  (if you don't have FUSE v. 3 in your system, use: sudo apt install fuse)\n  * CentOS/RHEL: sudo yum install fuse3\n  (if you don't have FUSE v. 3 in your system, use: sudo yum install fuse)\n\nBuilding MooseFS:\n  * Linux: Run ./linux_build.sh.\n  * FreeBSD: Run ./freebsd_build.sh.\n  * macOS: Run ./macosx_build.sh (note: macFUSE must be installed beforehand).\nThese scripts only build MooseFS and do not install binaries (i.e., they do not run `make install`). To install it, run `make install` (as root or `sudo make install`) manually.\n\nMinimal setup\n-------------\n\nJust three steps to have MooseFS up and running:\n\n1. Install at least one Master Server\n\n   1. Install moosefs-master package\n   2. Prepare default config (as root):\n        cd /etc/mfs\n        cp mfsmaster.cfg.sample mfsmaster.cfg\n        cp mfsexports.cfg.sample mfsexports.cfg\n   3. Prepare the metadata file (as root):\n        cd /var/lib/mfs\n        cp metadata.mfs.empty metadata.mfs\n        chown mfs:mfs metadata.mfs\n        rm metadata.mfs.empty\n   4. Run Master Server (as root): mfsmaster start\n   5. Make this machine visible under 'mfsmaster' name, e.g. by adding\n      a DNS entry (recommended) or by adding it in /etc/hosts on all servers\n      that run any of MooseFS components\n\n\n2. Install at least two Chunkservers\n\n   1. Install moosefs-chunkserver package\n   2. Prepare default config (as root):\n        cd /etc/mfs\n        cp mfschunkserver.cfg.sample mfschunkserver.cfg\n        cp mfshdd.cfg.sample mfshdd.cfg\n   3. At the end of mfshdd.cfg file make one or more entries containing paths\n      to HDDs / partitions designated for storing chunks, e.g.:\n        /mnt/chunks1\n        /mnt/chunks2\n        /mnt/chunks3\n      It is recommended to use XFS as an underlying filesystem\n      for disks designated to store chunks.\n   4. Change the ownership and permissions to mfs:mfs to above\n      mentioned locations:\n        chown mfs:mfs /mnt/chunks1 /mnt/chunks2 /mnt/chunks3\n        chmod 770 /mnt/chunks1 /mnt/chunks2 /mnt/chunks3\n   5. Start the Chunkserver: mfschunkserver start\n\n   Repeat the steps above for second (third, ...) Chunkserver.\n   More than two Chunkservers are strongly recommended.\n\n\n3. Client side: mount MooseFS filesystem\n\n   1. Install moosefs-client package\n   2. Mount MooseFS (as root):\n        mkdir /mnt/mfs\n        mount -t moosefs mfsmaster: /mnt/mfs\n      or:\n        mfsmount -H mfsmaster /mnt/mfs\n   3. You can also add an /etc/fstab entry to mount MooseFS during\n      the system boot:\n        mfsmaster: /mnt/mfs moosefs defaults,mfsdelayedinit 0 0\n\n   There are more configuration parameters available but most\n   of them may stay with defaults. We do our best to keep MooseFS easy\n   to deploy and maintain.\n\n   MooseFS, for testing purposes, can even be installed on a single machine!\n\n\nAdditional tools\n----------------\n\nSetting up moosefs-cli or moosefs-cgi with moosefs-cgiserv is also\nrecommended – it gives you a possibility to monitor the cluster online:\n  1. Install moosefs-cli moosefs-cgi moosefs-cgiserv packages\n     (they are typically set up on the Master Server)\n  2. Run MooseFS CGI Server (as root): mfscgiserv start\n  3. Open http://mfsmaster:9425 in your web browser\n\nIt is also strongly recommended to set up at least one Metalogger on a different\nmachine than Master Server (e.g. on one of Chunkservers). Metalogger constantly\nsynchronizes and backups the metadata:\n  1. Install moosefs-metalogger package\n  2. Prepare default config (as root):\n       cd /etc/mfs\n       cp mfsmetalogger.cfg.sample mfsmetalogger.cfg\n  3. Run Metalogger (as root): mfsmetalogger start\n\nRefer to installation guides (https://moosefs.com/support/#documentation)\nfor more details.\n\n\nSome facts\n==========\n\n  * Date of the first public release: 2008-05-30\n  * The project web site: https://moosefs.com\n  * Installation and using MooseFS: https://moosefs.com/support\n  * (Old) Sourceforge project site: https://sourceforge.net/projects/moosefs\n\n\nContact us\n==========\n\n  * Reporting bugs: GitHub issue (https://github.com/moosefs/moosefs/issues)\n    or support@moosefs.com\n  * General: contact@moosefs.com\n\n\nCopyright\n=========\n\nCopyright (c) 2008-2024 Jakub Kruszona-Zawadzki, Saglabs SA\n\nThis file is part of MooseFS.\n\nMooseFS is free software; you can redistribute it and/or modify it under\nthe terms of the GNU General Public License as published by the Free Software\nFoundation, version 2 (only).\n\nMooseFS is distributed in the hope that it will be useful, but WITHOUT ANY\nWARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS\nFOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License along with\nMooseFS; if not, write to the Free Software Foundation, Inc., 51 Franklin St,\nFifth Floor, Boston, MA 02111-1301, USA or visit\nhttp://www.gnu.org/licenses/gpl-2.0.html.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmoosefs%2Fmoosefs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmoosefs%2Fmoosefs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmoosefs%2Fmoosefs/lists"}