https://github.com/nextml-code/rkale

https://github.com/nextml-code/rkale

Last synced: 1 day ago
JSON representation

• Host: GitHub
• URL: https://github.com/nextml-code/rkale
• Owner: nextml-code
• Created: 2021-04-08T10:38:42.000Z (almost 4 years ago)
• Default Branch: master
• Last Pushed: 2024-05-08T08:24:16.000Z (8 months ago)
• Last Synced: 2024-08-08T18:11:11.983Z (5 months ago)
• Language: Python
• Size: 39.1 KB
• Stars: 3
• Watchers: 3
• Forks: 0
• Open Issues: 1
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# rkale

## Install

Install rkale in your project using poetry:

```bash

poetry add rkale

```

Use pip if you want a global installation:

```bash

pip install rkale

```

## Configuration

### Global

`~/.config/rkale/rkale.conf`:

```toml

[data]

root = "path to data folder where datasets are stored"

[aliases]

wasabi = "optional alias for remote in rclone.conf"

[rclone] # global flags for rclone

flags = ["--transfers 32", "--checkers 32"]

```

If aliases are empty the remote name from the project config is used in the

rclone lookup.

### Project

Configure project datasets in the pyproject.toml file:

`/pyproject.toml`:

```toml

[[tool.rkale.dataset]]

name = "dataset_1"

remote = "remote_1"

[[tool.rkale.dataset]]

name = "dataset_2"

remote = "remote_2"

```

The remote specified for the dataset must match a remote in the `rclone.conf`

or an alias in the global rkale configuration.

## Usage

### Python interface

```python

from rkale.config import dataset_paths

def dataset_path():

    return dataset_paths()["dataset_1"]

```

### Syncing datasets

Syncs the local datasets to be identical to the remote

```bash

rkale psync

```

Syncs the remote datasets to be identical to the local

```bash

rkale psync --upstream

```

Same as rclone sync but checks differences first and asks for confirmation

```bash

rkale sync  

```