{"id":13815583,"url":"https://github.com/oxalica/orb","last_synced_at":"2025-08-19T22:13:25.126Z","repository":{"id":229362425,"uuid":"744296081","full_name":"oxalica/orb","owner":"oxalica","description":"OneDrive as a block device","archived":false,"fork":false,"pushed_at":"2025-07-28T00:03:13.000Z","size":703,"stargazers_count":23,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-08-17T00:28:09.311Z","etag":null,"topics":["blockdevice","cloud","linux","onedrive"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oxalica.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-GPL-3.0","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"patreon":"oxalica"}},"created_at":"2024-01-17T02:11:45.000Z","updated_at":"2025-07-28T00:03:16.000Z","dependencies_parsed_at":"2024-04-24T07:20:58.668Z","dependency_job_id":"de2833dd-ef4f-49ec-ac78-5322583baa03","html_url":"https://github.com/oxalica/orb","commit_stats":null,"previous_names":["oxalica/orb"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/oxalica/orb","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxalica%2Forb","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxalica%2Forb/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxalica%2Forb/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxalica%2Forb/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oxalica","download_url":"https://codeload.github.com/oxalica/orb/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oxalica%2Forb/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271231032,"owners_count":24723034,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-19T02:00:09.176Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["blockdevice","cloud","linux","onedrive"],"created_at":"2024-08-04T04:03:37.431Z","updated_at":"2025-08-19T22:13:25.120Z","avatar_url":"https://github.com/oxalica.png","language":"Rust","funding_links":["https://patreon.com/oxalica"],"categories":["Rust"],"sub_categories":[],"readme":"# OneDrive as a block device\n\n:warning: This project is in beta stage.\n\n:warning: It is observed that in-kernel deadlock can occasionally happen in\n`orb-ublk` tests, causing any process calling `sync` stuck in `D` state forever,\nand the only solution is to hard-reset the system (you cannot even gracefully\nshutdown. `kill -9` does not work either). The exact reason is not yet known. It\nis not observed in `orb` production yet.\n\n## Audience\n\nIf you are not sure whether this project fits your need, then it does not. You\nare probably looking for\n[OneDrive Online](https://onedrive.live.com/) or sync and FUSE implementations\nlike [rclone](https://github.com/rclone/rclone).\n\nThis project may be helpful for :penguin: *real nerds* :penguin: who enjoy\nwacky block device stacking, intend to leverage block level encryption or their\nexisting BTRFS backup infrastructure, or explore fresh new bugs in BTRFS zoned\nmode, with the cost of *everything*.\n\n## Installation\n\nSystem requirements:\n\n- Linux \u003e= 5.19 is required for io-uring with `IORING_SETUP_SQE128` support.\n\n- Kernel driver `ublk_drv` and zoned block device support should be enabled.\n  Most distributions like Arch Linux and NixOS unstable meet these requirements\n  by default. You can check your system by:\n\n  ```console\n  $ zgrep -E 'CONFIG_BLK_DEV_UBLK|CONFIG_BLK_DEV_ZONED' /proc/config.gz\n  CONFIG_BLK_DEV_ZONED=y\n  CONFIG_BLK_DEV_UBLK=m\n  ```\n  If you see the same result, your kernel is probably supported.\n\n- You may need to run `sudo modprobe ublk_drv` manually to load the driver\n  first. This is not required for running orb in the shipped systemd service or\n  via NixOS module, which does this automatically.\n\n### Nix/NixOS (flake)\n\nThis project is packaged in Nix flake. Here's the simplified output graph:\n```\n├───nixosModules\n│   ├───default: Alias to `orb`.\n│   └───orb: The NixOS module.\n└───packages\n    ├───x86_64-linux\n    │   ├───default: Alias to `orb`.\n    │   ├───orb: The main program with systemd units.\n    │   ├───cryptsetup-format-zoned: workaround script for cryptsetup-luksFormat on zoned devices.\n    │   └───ublk-chown-unprivileged: The optional utility for unprivileged ublk.\n    [..more Linux platforms are supported..]\n```\n\n\u003cdetails\u003e\n\n\u003csummary\u003eExample configurations\u003c/summary\u003e\n\nTo use the orb service, add the flake input `github:oxalica/orb`, and import\nits NixOS modules.\n```nix\n# Example flake.nix for demostration. Please edit your own one to add changes.\n{\n  inputs.nixpkgs.url = \"github:NixOS/nixpkgs/nixos-unstable\";\n  inputs.orb.url = \"github:oxalica/orb\";\n\n  outputs = { nixpkgs, orb, ... }: {\n    nixosConfigurations.your-system = nixpkgs.lib.nixosSystem {\n      system = \"x86_64-linux\";\n      modules = with nixosModules; [\n        orb.nixosModules.orb\n        ./path/to/your/configuration.nix\n      ];\n    };\n  };\n}\n```\n\nNow you can use the module in your `configuration.nix`:\n```nix\n{ ... }:\n{\n  services.orb.instances = {\n    # The instance name. It coresponds to the systemd service\n    # `orb@my-device.service`. By default it will not be automatically started.\n    \"my-device\".settings = {\n        # Required device id. It's recommended to start at 80.\n        # This creates block device `/dev/ublkb80`.\n        ublk.id = 80; \n        # Other settings and their defaults can be seen in\n        # ./contrib/config-onedrive.example.toml\n        device = {\n          dev_size = \"1TiB\";\n          zone_size = \"256MiB\";\n          min_chunk_size = \"1MiB\";\n          max_chunk_size = \"256MiB\";\n        };\n        backend.onedrive.remote_dir = \"/orb\";\n    };\n  };\n\n  # If you want to mount the block device, you can create systemd mounts.\n  # This is an example.\n  systemd.mounts = [\n    {\n      type = \"btrfs\";\n      # Fill in your filesystem UUID after mkfs.\n      what = \"/dev/disk/by-uuid/11111111-2222-3333-4444-555555555555\";\n      where = \"/mnt/my-mount-point\";\n      # Do not forget dependencies.\n      requires = [ \"orb@my-device.service\" ];\n      after = [ \"orb@my-device.service\" ];\n      # It's recommended to set `noatime` and `compress` to reduce write\n      # frequency and amplification.\n      options = \"noatime,compress=zstd:7\";\n    }\n  ];\n}\n```\n\nNote that the service can only work after login and setup first. See the\nfollowing sections for details.\n\n\u003c/details\u003e\n\n### Other Linux distributions\n\nYou need following dependencies to be installed with your package manager:\n- Rust \u003e= 1.76\n- pkg-config\n- openssl\n\nBuild command: `cargo build --release`\n\n[`contrib/orb@.example.service`](./contrib/orb@.example.service)\nis the example template systemd service to install.\nThe instance configurations locate at `/etc/orb/\u003cname\u003e.toml`, whose format is\ndocumented in\n[`./contrib/config-onedrive.example.toml`](./contrib/config-onedrive.example.toml).\nOnce configured and logined (see the next section), run\n`systemctl start orb@\u003cname\u003e.service` to start the service.\n\n## First time login\n\nThe service configuration does not contain the login credential. It must be\ninteractively setup for the first time, and then the service will rotate the\ncredentials automatically unless the user revokes the permission, or after a\nlong offline time (seems to be \u003e1month, but is determined by Microsoft).\n\n1.  First, you need to know this project (orb) is an third party program which\n    access your files on Microsoft OneDrive on behalf of you, to provide block\n    device interface as a service. Your files and/or data on your Microsoft\n    OneDrive may be lost due to program bugs or other reasons. We provide no\n    warranties. By following the login steps below, you understood and want to\n    use orb at your own risk.\n\n2.  We cannot provide an \"official App/Client ID\" without risking impersonated\n    because this project is open sourced and free to distribute. So you need to\n    [register your own App on Microsoft\n    Azure](https://portal.azure.com/#view/Microsoft_AAD_RegisteredApps/ApplicationsListBlade).\n\n    In the registration page, \n    - In \"Supported account types\" section, select \"Personal Microsoft accounts\n      only\". Other accounts are currently unsupported.\n    - In \"Redirect URI (optional)\" section, select \"Public client/native\n      (mobile \u0026 desktop)\", and enter the following URI:\n      ```text\n      http://localhost\n      ```\n      It must be this exactly (it's `http` not `https`), or you may fail the\n      next step.\n\n    Then click \"Register\", it will jump to the registered App information page\n    if success. In \"Essential\" section, copy the UUID in the \"Application\n    (client) ID\" field. This is the Client ID to be used in the next step.\n    Note that one App can be used in multiple accounts, for multiple times. You\n    do not need to register more than one App in almost any cases.\n\n3.  Login with this command with root permission with arguments filled:\n    ```console\n    # orb login --systemd \u003cinstance\u003e --client-id \u003cthe-client-uuid-from-the-last-step\u003e\n    ```\n    `\u003cinstance\u003e` is the instance name of your systemd service (for example, you\n    setup `/etc/orb/foo.toml`, then `foo` is the instance name) or in NixOS\n    module setting `services.orb.instances.\u003cinstance\u003e`.\n\n    It will prompt a URL, and you need to open it in your browser and following\n    the interactive login steps to login into your Microsoft account with\n    OneDrive.\n\n    The credential will be saved under `/var/lib/orb/\u003cinstance\u003e`, owned by\n    root, and cannot be accessed by non-root users. It will be rotated by the\n    service, and please never copy or save it outside the local machine. If you\n    need to login to the same account on two machines, login twice.\n\n    :warning:\n    You must not serve the same remote directory simultaneously in multiple\n    instances (or machines), or it will cause data race and your data will be\n    corrupted. orb will try its best to detect and prevent such racing serving.\n\n4.  On success, the web page will redirect to a mostly empty page with only one line:\n    ```text\n    Successfully logined. This page can be closed.\n    ```\n\n    The command should exit normally with credential saved. Now you are ready\n    to start the orb service.\n   \n## Use the emulated block device\n\nOnce your logined and started the service successfully, you are ready to use it.\nUsually you need to create an filesystem on the emulated block device, and this\nis almost the same as the setup for your fresh hard disks, with a few\nexceptions:\n\n- The emulated device is under `/dev/ublkb\u003cID\u003e` where `ID` is specified in\n  your configuration `ublk.id`.\n\n- The device is a\n  [zoned device](https://zonedstorage.io/docs/introduction/zoned-storage)\n  (aka. ZBC/ZBD/ZNS, host managed SMR disks) due to API restrictions and\n  performance reasons. Only a few filesystems and/or device mappers support it,\n  eg. dm-crypt, F2FS and BTRFS.\n\n- It has a high latency and low throughput depend on your network. Doing\n  active works on it should be avoided. It can be used, for example, for\n  backup purpose.\n\n- :warning: Since the block device is emulated, you must ensure to `umount` the\n  filesystem on it before shutting down the backing device service\n  (`orb@\u003cinstance\u003e.service`), or you will lose your last written data. This\n  could be enforced by systemd mounts with a `BindsTo=` dependency.\n\n### Caveats on deletion and space usage\n\nDue to the limitation of OneDrive API, permanently deletion cannot be done via\nAPI. You may need to regularily \"Empty recycle bin\" on [OneDrive\nonline](https://onedrive.live.com) to free the capacity occupied.\n\n:warning: You MUST not \"Restore\" any files under the directory managed by the\norb service (`backend.onedrive.remote_dir`). Otherwise, it may break filesystem\nconsistency and your data may be lost.\n\n### Example: setup encryption via LUKS/dm-crypt\n\n\u003cdetails\u003e\n\u003csummary\u003e\nDetails\n\u003c/summary\u003e\n\n:warning: cryptsetup does not and probably will not support zoned devices\nnatively, because of non-trivial handling logic, see\n[this issue](https://gitlab.com/cryptsetup/cryptsetup/-/issues/877) and\n[this merge request](https://gitlab.com/cryptsetup/cryptsetup/-/merge_requests/638).\nGenerally you should avoid this unsupported usage, unless there is no other way\naround.\n\n:warning: Of course, this will destroy all of your data on the emulated device,\naka. the remote directory in OneDrive holding the data.\n\ncryptsetup does not support formatting zoned devices, but dm-crypt supports it.\nWe need to format and place the LUKS2 header manually, and then it can be\nopened and/or closed in the normal way. For convenience, there is a script\nunder\n[`./contrib/cryptsetup-format-zoned.sh`](./contrib/cryptsetup-format-zoned.sh)\nto mimic `cryptsetup luksFormat` as a workaround. Run:\n\n```console\n# ./contrib/cryptsetup-format-zoned.sh /dev/ublkb\u003cID\u003e # Use a a password.\nOR\n# ./contrib/cryptsetup-format-zoned.sh /dev/ublkb\u003cID\u003e /path/to/key/file # Use a key file.\n```\n\nAlternatively, you can run the script via flake package:\n```console\n$ nix shell github:oxalica/orb#cryptsetup-format-zoned -c sudo cryptsetup-format-zoned /dev/ublkb\u003cID\u003e\n```\n\nNote that editing header, ie. adding or removing keys, also requires careful\nmanual operations. You need do it yourself when needed.\n\nAfter formatting the block device, you can open and/or close it in the normal\nway:\n```console\n# cryptsetup luksOpen /dev/ublkb\u003cID\u003e my-device-unencrypted\n# cryptsetup close my-device-unencrypted\n```\n\nIf you are using key files, you can also use systemd-cryptsetup services to\nmanage dm-crypt. This is useful when you want to specify dependencies to\n`orb@\u003cinstance\u003e.service` and downstream services, eg. backup services.\n```nix\n{ ... }:\n{\n  environment.etc.\"crypttab\".text = ''\n    mydecrypteddev /dev/ublkb\u003cID\u003e /path/to/key/file noauto\n  '';\n  systemd.services.\"systemd-cryptsetup@mydecrypteddev\" = {\n    # Inform Nix that this is an overriding units for auto-generated ones.\n    overrideStrategy = \"asDropin\";\n    # Specify dependencies to the orb service.\n    bindsTo = [ \"orb@my-instance.service\" ];\n    after = [ \"orb@my-instance.service\" ];\n  };\n}\n```\n\n\u003c/details\u003e\n\n### Example: format it as BTRFS\n\n\u003cdetails\u003e\n\u003csummary\u003e\nDetails\n\u003c/summary\u003e\n\n:warning: Of course, this will destroy all of your data on the emulated device,\naka. the remote directory in OneDrive holding the data.\n\nIt is recommended to format BTRFS with `block-group-tree` feature enabled, to\ndramastically reduce mounting time (~50s to ~2s). You need btrfs-progs \u003e= 6.8.1\nwith [a relevant bug](https://github.com/kdave/btrfs-progs/issues/765) getting fixed.\n\n```console\n# mkfs.btrfs /dev/ublkb\u003cID\u003e -O block-group-tree\n```\n\n`zoned` feature will be automatically detected and enabled without manual\nspecification.\n\nNow you can mount it and do read/write operations. These are recommended mount\noptions (disable atime, high level zstd compression enabled):\n```console\nsudo mount -t btrfs -o noatime,compress=zstd:7 /dev/ublkb\u003cID\u003e /mnt/my-mount-point\n```\n\n\u003c/details\u003e\n\n## License\n\nThe sub-package `orb-ublk` and `ublk-chown-unprivileged` (directory\n`/orb-ublk`, `/ublk-chown-unprivileged` and the whole sub-tree of them)\nare licensed under either of [Apache License, Version\n2.0](./orb-ublk/LICENSE-APACHE) or [MIT license](./orb-ublk/LICENSE-MIT) at\nyour option.\n\nThe main package (all other files in the repository except content of\n`/orb-ublk` and/or `/ublk-chown-unprivileged` directory) is licensed under\n[GNU General Public License v3.0](./LICENSE-GPL-3.0) or (at your option) later\nversions.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foxalica%2Forb","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foxalica%2Forb","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foxalica%2Forb/lists"}