{"id":27185037,"url":"https://github.com/threefoldtech/rfs","last_synced_at":"2026-03-27T04:08:55.969Z","repository":{"id":37091661,"uuid":"172350020","full_name":"threefoldtech/rfs","owner":"threefoldtech","description":"a dedupe filesystem in rust","archived":false,"fork":false,"pushed_at":"2025-12-07T14:31:06.000Z","size":4586,"stargazers_count":3,"open_issues_count":15,"forks_count":1,"subscribers_count":2,"default_branch":"development","last_synced_at":"2026-02-18T12:48:10.257Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/threefoldtech.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2019-02-24T14:48:33.000Z","updated_at":"2025-10-16T12:53:26.000Z","dependencies_parsed_at":"2023-11-06T11:38:12.376Z","dependency_job_id":"ac7632ab-cf58-4736-8cfa-68c97b15ff71","html_url":"https://github.com/threefoldtech/rfs","commit_stats":null,"previous_names":["threefoldtech/mycofs","threefoldtech/rfs"],"tags_count":27,"template":false,"template_full_name":null,"purl":"pkg:github/threefoldtech/rfs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/threefoldtech%2Frfs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/threefoldtech%2Frfs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/threefoldtech%2Frfs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/threefoldtech%2Frfs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/threefoldtech","download_url":"https://codeload.github.com/threefoldtech/rfs/tar.gz/refs/heads/development","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/threefoldtech%2Frfs/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31018555,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-27T03:51:26.850Z","status":"ssl_error","status_checked_at":"2026-03-27T03:51:09.693Z","response_time":164,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-04-09T17:10:10.371Z","updated_at":"2026-03-27T04:08:55.952Z","avatar_url":"https://github.com/threefoldtech.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# RFS - Remote File System\n\n[![Test](https://github.com/threefoldtech/rfs/actions/workflows/tests.yaml/badge.svg?branch=master)](https://github.com/threefoldtech/rfs/actions/workflows/tests.yaml)\n\n## What is RFS?\n\nRFS (Remote File System) is a powerful command-line tool that lets you efficiently store, distribute, and access filesystems across different storage backends. It solves common problems in file distribution by separating metadata from content, allowing for:\n\n- **Efficient file distribution** - Share only the metadata, download content on-demand\n- **Reduced bandwidth usage** - Only download the files you actually need\n- **Flexible storage options** - Store content in local directories, ZDB, S3, or HTTP backends\n- **Docker image conversion** - Convert Docker images to lightweight, mountable filesystems\n- **Web-based management** - Manage your filesystems through a user-friendly web interface\n\n## Key Features\n\n- **Flists**: Compact metadata files that describe filesystems without containing the actual data\n- **On-demand access**: Files are only downloaded when accessed, saving bandwidth and storage\n- **Content deduplication**: Identical files are stored only once, even across different flists\n- **Multiple storage backends**: Store content in directory, ZDB, S3, or HTTP backends\n- **Docker conversion**: Convert Docker images to flists for efficient distribution\n- **Server functionality**: Run a server for web-based flist management\n- **Sharding and replication**: Distribute and replicate content across multiple storage backends\n\n## Common Use Cases\n\n### Efficient Docker Image Distribution\n\nConvert Docker images to flists for more efficient distribution and usage:\n\n```bash\n# Convert a Docker image to an flist\nrfs docker -i nginx:latest -s dir:///tmp/store\n\n# Mount the resulting flist\nsudo rfs mount -m nginx-latest.fl /mnt/nginx\n```\n\n### Distributing Large Filesystems\n\nPackage and distribute large filesystems efficiently:\n\n```bash\n# Create an flist from a directory\nrfs pack -m myapp.fl -s dir:///tmp/store /path/to/myapp\n\n# Share the flist (typically \u003c1MB) instead of the entire content\n# Recipients can mount it and access files on-demand\nsudo rfs mount -m myapp.fl /mnt/myapp\n```\n\n### Web Content Distribution\n\nPublish and distribute web content efficiently:\n\n```bash\n# Create an flist from a website directory\nrfs pack -m website.fl -s s3://user:pass@s3.example.com:9000/bucket /path/to/website\n\n# Mount the website on a server\nsudo rfs mount -m website.fl /var/www/html\n```\n\n### Application Deployment\n\nDeploy applications with all their dependencies:\n\n```bash\n# Convert an application Docker image to an flist\nrfs docker -i myapp:latest -s zdb://zdb.example.com:9900/namespace\n\n# Deploy the application on multiple servers by mounting the flist\nsudo rfs mount -m myapp-latest.fl /opt/myapp\n```\n\n## Quick Start\n\n### Installation\n\n```bash\n# Install dependencies\nsudo apt-get install -y build-essential fuse libfuse-dev\n\n# Install Rust\ncurl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh\nsource $HOME/.cargo/env\n\n# Clone the repository\ngit clone https://github.com/threefoldtech/rfs.git\ncd rfs\n\n# Build RFS\nrustup target add x86_64-unknown-linux-musl\ncargo build --features build-binary --release --target=x86_64-unknown-linux-musl\n\n# Install the binary\nsudo cp ./target/x86_64-unknown-linux-musl/release/rfs /usr/local/bin/\n```\n\n### Creating Your First Flist\n\n```bash\n# Create a directory to use as a store\nmkdir -p ~/rfs-store\n\n# Create an flist from a directory\nrfs pack -m ~/test.fl -s dir://~/rfs-store ~/my-directory\n\n# Mount the flist\nmkdir -p ~/mount-point\nsudo rfs mount -m ~/test.fl ~/mount-point\n\n# Access the files\nls -la ~/mount-point\n\n# Unmount when done\nsudo umount ~/mount-point\n```\n\n### Converting a Docker Image\n\n```bash\n# Create a directory to use as a store\nmkdir -p ~/rfs-store\n\n# Convert a Docker image to an flist\nrfs docker -i alpine:latest -s dir://~/rfs-store\n\n# Mount the resulting flist\nmkdir -p ~/mount-point\nsudo rfs mount -m alpine-latest.fl ~/mount-point\n\n# Run commands in the mounted filesystem\nsudo chroot ~/mount-point /bin/sh -c \"ls -la\"\n\n# Unmount when done\nsudo umount ~/mount-point\n```\n\n### Running the Server\n\n```bash\n# Create a configuration file\ncat \u003e config.toml \u003c\u003c EOF\nhost = \"localhost\"\nport = 3000\nstore_url = [\"dir:///tmp/store\"]\nflist_dir = \"flists\"\n\njwt_secret = \"your-secret-key\"\njwt_expire_hours = 5\n\n[[users]]\nusername = \"admin\"\npassword = \"admin-password\"\nEOF\n\n# Create the flists directory\nmkdir -p flists/admin\n\n# Run the server\nrfs server --config-path config.toml\n```\n\n## Command Reference\n\nThe `rfs` command provides all the functionality you need to work with flists:\n\n- `rfs pack` - Create flists from directories\n- `rfs mount` - Mount flists as filesystems\n- `rfs unpack` - Extract flist contents to a directory\n- `rfs docker` - Convert Docker images to flists\n- `rfs server` - Run the RFS server for web-based management\n- `rfs config` - Manage flist metadata and stores\n- `rfs clone` - Copy data between stores\n- `rfs flist tree` - Display flist contents as a tree structure\n- `rfs flist inspect` - Inspect file details within an flist\n- And more...\n\nFor detailed information about each command, use the `--help` flag:\n\n```bash\nrfs --help\nrfs pack --help\nrfs mount --help\n```\n\n## Documentation\n\nComprehensive documentation is available in the [docs](./docs) directory:\n\n### Getting Started\n\n- [Installation and Basic Usage](./docs/tutorials/getting-started.md)\n### Core Functionality\n\n- [Getting Started](./docs/tutorials/getting-started.md) - Installation and basic usage\n- [Creating Flists](./docs/tutorials/creating-flists.md) - How to create flists from directories\n- [Mounting Flists](./docs/tutorials/mounting-flists.md) - How to mount and use flists\n- [End-to-End Flist Workflow](./docs/tutorials/end-to-end-flist-workflow.md) - Complete workflow for creating and using flists\n\n### Docker Integration\n\n- [Converting Docker Images](./docs/tutorials/docker-conversion.md) - How to convert Docker images to flists\n- [End-to-End Docker Workflow](./docs/tutorials/end-to-end-docker-workflow.md) - Complete workflow for Docker conversion\n\n### Server and Distribution\n\n- [Server Setup](./docs/tutorials/server-setup.md) - How to set up the RFS server\n- [Website Publishing](./docs/tutorials/website-publishing.md) - How to publish websites using RFS\n- [Syncing Files](./docs/tutorials/syncing-files.md) - How to sync files between RFS servers\n\n### User Guides\n\n- [RFS Command Reference](./docs/user-guides/rfs-cli.md)\n- [RFS Server Setup and Usage](./docs/user-guides/fl-server.md)\n- [Web Interface Guide](./docs/user-guides/frontend.md)\n- [Performance Tuning](./docs/user-guides/performance-tuning.md)\n- [Troubleshooting](./docs/user-guides/troubleshooting.md)\n\n### Concepts and Architecture\n\n- [Understanding Flists](./docs/concepts/flists.md)\n- [Storage Backends](./docs/concepts/stores.md)\n- [Caching](./docs/concepts/caching.md)\n- [Sharding and Replication](./docs/concepts/sharding.md)\n- [System Architecture](./docs/architecture/overview.md)\n\n## Community and Support\n\n- **GitHub Issues**: Report bugs or request features on our [GitHub repository](https://github.com/threefoldtech/rfs/issues)\n- **Documentation**: Comprehensive documentation is available in the [docs](./docs) directory\n\n## License\n\nThis project is licensed under the [Apache License 2.0](./LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthreefoldtech%2Frfs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthreefoldtech%2Frfs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthreefoldtech%2Frfs/lists"}