{"id":13400882,"url":"https://github.com/ovh/svfs","last_synced_at":"2025-03-14T06:32:00.137Z","repository":{"id":57500997,"uuid":"51554547","full_name":"ovh/svfs","owner":"ovh","description":"The Swift Virtual File System","archived":true,"fork":false,"pushed_at":"2021-03-15T15:21:08.000Z","size":1925,"stargazers_count":374,"open_issues_count":34,"forks_count":54,"subscribers_count":30,"default_branch":"master","last_synced_at":"2024-09-30T21:36:17.850Z","etag":null,"topics":["cloud-storage","filesystem","fuse","go","golang","hubic","openstack-swift","ovh"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ovh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-02-11T23:17:29.000Z","updated_at":"2024-07-16T11:44:44.000Z","dependencies_parsed_at":"2022-08-31T03:21:23.221Z","dependency_job_id":null,"html_url":"https://github.com/ovh/svfs","commit_stats":null,"previous_names":["xlucas/svfs"],"tags_count":25,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ovh%2Fsvfs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ovh%2Fsvfs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ovh%2Fsvfs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ovh%2Fsvfs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ovh","download_url":"https://codeload.github.com/ovh/svfs/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243537950,"owners_count":20307098,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloud-storage","filesystem","fuse","go","golang","hubic","openstack-swift","ovh"],"created_at":"2024-07-30T19:00:56.645Z","updated_at":"2025-03-14T06:32:00.125Z","avatar_url":"https://github.com/ovh.png","language":"Go","readme":"# *** This project is not maintained anymore ***\n\n# The Swift Virtual File System\n\n[![Release](https://badge.fury.io/gh/ovh%2Fsvfs.svg)](https://github.com/ovh/svfs/releases)\n[![Github All Releases](https://img.shields.io/github/downloads/ovh/svfs/total.svg)](https://github.com/ovh/svfs/releases)\n[![Build Status](https://travis-ci.org/ovh/svfs.svg?branch=master)](https://travis-ci.org/ovh/svfs)\n[![Go Report Card](https://goreportcard.com/badge/github.com/ovh/svfs)](https://goreportcard.com/report/github.com/ovh/svfs)\n[![Coverage Status](https://coveralls.io/repos/github/ovh/svfs/badge.svg?branch=master)](https://coveralls.io/github/ovh/svfs?branch=master)\n[![GoDoc](https://godoc.org/github.com/ovh/svfs/svfs?status.svg)](https://godoc.org/github.com/ovh/svfs/svfs)\n\n**SVFS** is a Virtual File System over Openstack Swift built upon fuse. It is compatible with [hubiC](https://hubic.com),\n[OVH Public Cloud Storage](https://www.ovh.com/fr/cloud/storage/object-storage) and basically every endpoint using a standard\nOpenstack Swift setup.  It brings a layer of abstraction over object storage, making it as accessible and convenient as a\nfilesystem, without being intrusive on the way your data is stored.\n\n## Disclaimer\nThis is not an official project of the Openstack community.\n\n## Installation\n\nDownload and install the latest [release](https://github.com/ovh/svfs/releases) packaged for your distribution.\n\n## Usage\n\n#### Mount command\n\nOn Linux (requires fuse and ruby) :\n\n```\nmount -t svfs -o \u003coptions\u003e \u003cdevice\u003e /mountpoint\n```\n\nOn OSX (requires osxfuse and ruby) :\n\n```\nmount_svfs \u003cdevice\u003e /mountpoint -o \u003coptions\u003e\n```\n\nNotes :\n- You can pick any name you want for the `device` parameter.\n- All available mount options are described later in this document.\n\nCredentials can be specified in mount options, however this may be desirable to read them from an external source. The following sections desribe alternative approaches.\n\n#### Reading credentials from the environment\n\nSVFS supports reading the following set of environment variables :\n\n* If you are using HubiC :\n```\n HUBIC_AUTH\n HUBIC_TOKEN\n```\n* If you are using a vanilla Swift endpoint (like OVH PCS), after sourcing your [OpenRC](http://docs.openstack.org/user-guide/common/cli-set-environment-variables-using-openstack-rc.html) file :\n```\n OS_AUTH_URL\n OS_USERNAME\n OS_PASSWORD\n OS_REGION_NAME\n OS_TENANT_NAME\n```\n* If you already authenticated to an identity endpoint :\n```\n OS_AUTH_TOKEN\n OS_STORAGE_URL\n```\n\n#### Reading credentials from a configuration file\n\nAll environment variables can also be set in a YAML configuration file placed at `/etc/svfs.yaml`.\n\nFor instance :\n```yaml\nhubic_auth: XXXXXXXXXX..\nhubic_token: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX...\n```\n\n## Usage with OVH products\n\n- Usage with OVH Public Cloud Storage is explained [here](docs/PCS.md).\n- Usage with hubiC is explained [here](docs/HubiC.md).\n\n## Mount options\n\n#### Keystone options\n\n* `auth_url`: keystone URL (default is https://auth.cloud.ovh.net/v2.0).\n* `username`: your keystone user name.\n* `password`: your keystone password.\n* `tenant`: your project name.\n* `region`: the region where your tenant is.\n* `version`: authentication version (`0` means auto-discovery which is the default).\n* `storage_url`: the storage endpoint holding your data.\n* `internal_endpoint`: the storage endpoint type (default is `false`).\n* `token`: a valid token.\n\nOptions `region`, `version`, `storage_url` and `token` are guessed during authentication if\nnot provided.\n\n#### Hubic options\n\n* `hubic_auth`: hubic authorization token as returned by the `hubic-application` command.\n* `hubic_times` : use file times set by hubic synchronization clients. Option `attr`\nshould also be set for this to work.\n* `hubic_token` : hubic refresh token as returned by the `hubic-application` command.\n\n#### Swift options\n\n* `container`: which container should be selected while mounting the filesystem. If not set,\nall containers within the tenant will be available under the chosen mountpoint.\n* `storage_policy`: expected containers storage policy. This is used to ignore containers\nnot matching a particular storage policy name. If empty, this setting is ignored (default).\n* `segment_size`: large object segments size in MB. When an object has a content larger than\nthis setting, it will be uploaded in multiple parts of the specified size. Default is 256 MB.\nSegment size should not exceed 5 GB.\n* `connect_timeout`: connection timeout to the swift storage endpoint. Default is 15 seconds.\n* `request_timeout`: timeout of requests sent to the swift storage endpoint. Default is 5 minutes.\n\n#### Prefetch options\n\n* `block_size`: Filesystem block size in bytes. This is only used to report correct `stat()` results.\n* `readahead_size`: Readahead size in KB. Default is 128 KB.\n* `readdir`: Overall concurrency factor when listing segmented objects in directories (default is 20).\n* `attr`: Handle base attributes.\n* `xattr`: Handle extended attributes.\n* `transfer_mode`: Enforce network transfer optimizations. The following flags / features can be combined :\n - `1` : disable explicit empty file creation.\n - `2` : disable explicit directory creation.\n - `4` : disable directory content check on removal.\n - `8` : disable file check in read only opening.\n\n#### Cache options\n\n* `cache_access`: cache entry access count before refresh. Default is -1 (unlimited access).\n* `cache_entries`: maximum entry count in cache. Default is -1 (unlimited).\n* `cache_ttl`: cache entry timeout before refresh. Default is 1 minute.\n\n#### Access restriction options\n\n* `allow_other`: Bypass `allow_root`.\n* `allow_root`: Restrict access to root and the user mounting the filesystem.\n* `default_perm`: Restrict access based on file mode (useful with `allow_other`).\n* `uid`: default files uid (defaults to current user uid).\n* `gid`: default files gid (defaults to current user gid).\n* `mode`: default files permissions (default is 0700).\n* `ro`: enable read-only access.\n\n#### Debug options\n\n* `debug`: enable debug log.\n* `stdout` : stdout redirection expression (e.g. `\u003e/dev/null`).\n* `stderr` : stderr redirection expression (e.g. `\u003e\u003e/var/log/svfs.log`).\n* `profile_addr`: Golang profiling information will be served at this address (`ip:port`) if set.\n* `profile_cpu`: Golang CPU profiling information will be stored to this file if set.\n* `profile_ram`: Golang RAM profiling information will be stored to this file if set.\n\n#### Performance options\n* `go_gc`: set garbage collection target percentage. A garbage collection is triggered when the\nheap size exceeds, by this rate, the remaining heap size after the previous collection. A lower\nvalue triggers frequent GC, which means memory usage will be lower at the cost of higher CPU\nusage. Setting a higher value will let the heap size grow by this percent without collection,\nreducing GC frequency. A Garbage collection is forced if none happened for 2 minutes. Note that\nunused heap memory is not reclaimed after collection, it is returned to the operating system\nonly if it appears unused for 5 minutes.\n\n\n## Limitations\n\n**Be aware that SVFS doesn't transform object storage to block storage.**\n\nSVFS doesn't support :\n\n* Opening files in other modes than `O_CREAT`, `O_RDONLY` and `O_WRONLY`.\n* Moving directories.\n* Renaming containers.\n* SLO (but supports DLO).\n* Per-file uid/gid/permissions (but per-mountpoint).\n* Symlink targets across containers (but within the same container).\n\nTake a look at the [docs](docs) for further discussions about SVFS approach.\n\n## FAQ\n\nGot errors using `rsync` with svfs ? Can't change creation time ? Why svfs after all ?\n\nTake a look at the [FAQ](docs/FAQ.md).\n\n## Hacking\n\nMake sure to use the latest version of go and follow [contribution guidelines](CONTRIBUTING.md) of SVFS.\n\n## License\nThis work is under the BSD license, see the [LICENSE](LICENSE) file for details.\n","funding_links":[],"categories":["File System","Built On"],"sub_categories":["Kafka"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fovh%2Fsvfs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fovh%2Fsvfs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fovh%2Fsvfs/lists"}