Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/frankyang0529/longhorn-engine

World's smallest storage controller
https://github.com/frankyang0529/longhorn-engine

Last synced: about 2 months ago
JSON representation

World's smallest storage controller

Host: GitHub
URL: https://github.com/frankyang0529/longhorn-engine
Owner: FrankYang0529
License: apache-2.0
Fork: true (longhorn/longhorn-engine)
Created: 2021-10-18T03:04:29.000Z (about 3 years ago)
Default Branch: master
Last Pushed: 2024-06-14T02:38:47.000Z (5 months ago)
Last Synced: 2024-06-14T03:44:59.955Z (5 months ago)
Language: Python
Homepage:
Size: 20 MB
Stars: 0
Watchers: 2
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Codeowners: .github/CODEOWNERS

Awesome Lists containing this project

README

Longhorn Engine
==
[![Build Status](https://github.com/longhorn/longhorn-engine/actions/workflows/build.yml/badge.svg)](https://github.com/longhorn/longhorn-engine/actions/workflows/build.yml)[![Go Report Card](https://goreportcard.com/badge/github.com/rancher/longhorn-engine)](https://goreportcard.com/report/github.com/rancher/longhorn-engine)

Longhorn Engine implements a lightweight block device storage controller capable of storing the data in a number of replicas. It functions like a network RAID controller.

1. The replicas are backed by Linux sparse files, and support efficient snapshots using differencing disks.
1. The replicas function like a networked disk, supporting read/write operations over a network protocol.
1. The frontend (only Open-iSCSI/tgt are supported at this moment) is a kernel driver that translates read/write operations on the Longhorn block device (mapped at `/dev/longhorn/vol-name`) to user-level network requests on the controller.
1. Each Longhorn block device is backed by its own dedicated controller.
1. The controller synchronously replicates write operations to all replicas.
1. The controller detects faulty replicas and rebuilds replicas.
1. The controller coordinates snapshot and backup operations.
1. Controllers and replicas are packaged as Docker containers.

The following figure illustrates the relationship between the Longhorn block device, tgt frontend, controller, and replicas.

![Overview Graphics](/overview.png)

## Building from source code

`make`

## Running a controller with a single replica

The easiest way to try the Longhorn Engine is to start a controller with a single replica.

Host needs to have `docker` installed. Run the following command to make sure:
```
docker info
```

#### With tgt-blockdev frontend

User needs to make sure the host has `iscsiadm` installed. Run the following command to check:
```
iscsiadm --version
```

To start Longhorn Engine with an single replica, run the following command:
```
docker run --privileged -v /dev:/host/dev -v /proc:/host/proc -v /volume \
longhornio/longhorn-engine:master-head launch-simple-longhorn vol-name 10g tgt-blockdev
```

That will create the device `/dev/longhorn/vol-name`

## Running a controller with multiple replicas

In order to start Longhorn Engine with multiple replicas, you need to setup a network between replica container and controller container. Here we use Docker network feature to demonstrate that:

##### 1. Create a network named `longhorn-net`
```
docker network create --subnet=172.18.0.0/16 longhorn-net
```
##### 2. Add two replicas to the network, and set their IPs to `172.18.0.2` and `172.18.0.3`:
```
docker run --net longhorn-net --ip 172.18.0.2 -v /volume \
longhornio/longhorn-engine longhorn replica --listen 172.18.0.2:9502 --size 10g /volume
docker run --net longhorn-net --ip 172.18.0.3 -v /volume \
longhornio/longhorn-engine longhorn replica --listen 172.18.0.3:9502 --size 10g /volume
```

##### 3. Start the controller. Take tgt-blockdev for example:
```
docker run --net longhorn-net --privileged -v /dev:/host/dev -v /proc:/host/proc \
longhornio/longhorn-engine longhorn controller --frontend tgt-blockdev \
--replica tcp://172.18.0.2:9502 --replica tcp://172.18.0.3:9502 vol-name
```
Now you will have device `/dev/longhorn/vol-name`.

## Run `longhorn` command

The `longhorn` command allows you to manage a Longhorn controller. By executing the `longhorn` command in the controller container, you can list replicas, add and remove replicas, take snapshots, and create backups.

```
$ docker exec longhorn ls
ADDRESS MODE CHAIN
tcp://172.18.0.2:9502 RW [volume-head-000.img]
tcp://172.18.0.3:9502 RW [volume-head-000.img]
```
## Contribution

Please check [the main repo](https://github.com/longhorn/longhorn#community) for the contributing guide.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

[http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0)

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.