https://github.com/rubiojr/rclone-charmfs
CharmFS backend for Rclone
https://github.com/rubiojr/rclone-charmfs
charm charmfs rclone
Last synced: about 2 months ago
JSON representation
CharmFS backend for Rclone
- Host: GitHub
- URL: https://github.com/rubiojr/rclone-charmfs
- Owner: rubiojr
- Created: 2022-05-03T13:19:33.000Z (about 3 years ago)
- Default Branch: main
- Last Pushed: 2022-05-03T17:45:03.000Z (about 3 years ago)
- Last Synced: 2025-03-28T00:05:38.311Z (2 months ago)
- Topics: charm, charmfs, rclone
- Language: Go
- Homepage:
- Size: 27.3 KB
- Stars: 15
- Watchers: 3
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Rclone :heart: CharmFS
A [rclone](https://github.com/rclone/rclone) CharmFS backend to manage your [Charm Cloud](https://charm.sh) files.
The backend is currently highly experimental, in a working prototype stage, very inefficient and with some functionality broken.
**Don't use it with real/production data for now.**
## Building
Install [Go](https://go.dev) and Run `make`.
This will clone the rclone repository, add the CharmFS backend code and build the `rclone-charm` binary.
## Running
Note that you'll need a working Charm account first, with ssh keys generated etc.
Configure the remote firt:
```
rclone-charm config create charmfs charm url=https://cloud.charm.sh
```
If you want to use your own Charm server URL, it needs to be set using environment variables for now, as remote rclone config is currently ignored (not implemented):
```
export CHARM_HOST=my.charm.host
rclone-charm ls charmfs:
```
## Working commands
Far from being bug free and efficient, may eat data.
* cat
* copy
* delete
* ls, lsd, lsf, lsjson, lsl
* lsd
* mount (partially, VFS caching doesn't work as expected, which means opening files with other programs will be broken)
* tree
## Unsupported commands
* about (No quota support in CharmFS to my knowledge)
* size (no way to get remote plaintext file size currently)
* sha1sum (no way to get remote plaintext sha1sum currently)
* touch (Change remote file mod time not currently supported by CharmFS to my knowledge)
* selfupdate (the plan is to upstream the backend eventually)
## Commands not working
* mkdir (not sure yet if we can mkdir empty directories in Charm)
* move (not implemented)
* moveto (not implemented)
* deletefile (buggy)
## Commands not tested
Everything else.
## Planed
* Custom config endpoint support (supported via CHARM_* env vars currently)
* Unit tests
* Replace Bash integration tests with Go test
* Upstream submission
## Gotchas
CharmFS uses end-to-end encryption, which makes some Rclone file operations a bit more complicated or downright impossible. In particular, things like:
* Getting the remote (plaintext) file size without downloading it first
* Get the remote real file name
* Checksumming local and remote files,
* Comparing local and remote file sizes
and other file operations where plaintext access by the server is required will probably be never supported, unless CharmFS gains access to some additional file metadata eventually.
This is a good thing in my book, but it'll have a (significant perhaps) performance impact and limit Rclone functionality when using a CharmFS remote.
## Testing
Run `make test`.
This will:
* Clone the rclone repository
* Add the CharmFS backend code
* Build the rclone binary
* Download a `charm` binary
* Start a charm server locally
* Run a few integration tests