Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/golemfactory/ya-runtime-sdk
Rust library for building new computation environments and self-contained runtimes for yagna
https://github.com/golemfactory/ya-runtime-sdk
golem ya-runtime
Last synced: 4 months ago
JSON representation
Rust library for building new computation environments and self-contained runtimes for yagna
- Host: GitHub
- URL: https://github.com/golemfactory/ya-runtime-sdk
- Owner: golemfactory
- License: gpl-3.0
- Created: 2021-05-04T18:50:38.000Z (almost 4 years ago)
- Default Branch: main
- Last Pushed: 2023-06-05T08:16:18.000Z (over 1 year ago)
- Last Synced: 2024-10-01T04:29:43.937Z (4 months ago)
- Topics: golem, ya-runtime
- Language: Rust
- Homepage:
- Size: 130 KB
- Stars: 2
- Watchers: 11
- Forks: 3
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# ya-runtime-sdk
`ya-runtime-sdk` is a Rust library for building Computation Environments and Self-contained Runtimes,
executed by Provider nodes in the Golem network.
`ya-runtime-sdk` provides facilitation in the following areas:
- interfacing with the computation orchestrator
- computation lifecycle management
- command output handling
- billing information reporting (TBD)
---
Table of contents
- [Runtime overview](#runtime-overview)
- [Runtime orchestration](#runtime-orchestration)
- [Runtime execution mode](#runtime-execution-mode)
- [Command execution mode](#command-execution-mode)
- [Complementary functions](#complementary-functions)
- [Events](#events)
- [Implementation](#implementation)
- [Context](#context)
- [Configuration](#configuration)
- [Debugging](#debugging)
- [Deploying](#deploying)
---
## Runtime overview
Runtime is responsible for performing computation specified in an Agreement between a Provider and a
Requestor. Runtimes are executed only after a successful marketplace negotiation has taken place, where prices,
computation deadlines and rented hardware resources have been agreed upon.
The degree of remote control that a Requestor can have over a Runtime depends on its flavour:
1. Computation Environment
Fully controlled by the Requestor via a well-defined set of commands ("ExeScript").
Requestor is responsible for triggering deployment of the payload (e.g. a Virtual Machine image)
specified in an Agreement, starting the environment, executing commands within that environment
and terminating the setup.
2. Self-contained Runtime
The payload is pre-defined by the developer. In this case, deploying, starting, executing
commands and terminating the runtime can be only hinted by the Requestor; the implementation determines the exact
behaviour of those actions.
### Runtime orchestration
Runtimes are orchestrated by their parent process, the ExeUnit Supervisor, via a standardized set of command line
arguments and a Runtime API protocol. `ya-runtime-sdk` handles all the communication automatically for any struct
implementing the `Runtime` trait.
The `Runtime` trait specifies handlers for each of the execution phases:
- `deploy`
**Description:** Initial configuration of the environment and payload.
**Output:** Optional JSON response in the following format:
```json
{
"startMode": "blocking",
"valid": {"Ok": "success"},
"vols": [
{"name": "vol-9a0c1c4a", "path": "/in"},
{"name": "vol-a68672e0", "path": "/out"}
],
"customKey": "customValue"
}
```
**Required properties**:
- `startMode`
Determines whether a runtime is a long-running application or a one-shot command.
- `"blocking"` for `RuntimeMode::Server`
- `"empty"` for `RuntimeMode::Command`
- `valid`
Deployment status.
- `{"Ok": "success message"}`
- `{"Err": "error message"}`
- `vols`
Local filesystem directory to runtime directory mapping.
- `name` is a subdirectory on a local filesystem, in a directory chosen by the Supervisor,
- `path` is an alias of that directory, seen from inside the runtime and Supervisor services (e.g. file transfer)
- `start`
**Description:** Enable the runtime to be used by the Requestor.
Can be run in **one** of the [execution modes](#runtime-execution-mode).
**Output:** Optional JSON response.
- `run_command`
**Description:** Parse, interpret and handle an array of string arguments. Optional in case of self-contained runtimes.
Can be run in **any** of the [execution modes](#command-execution-mode)
- `kill_command`
**Description:** Terminate command execution.
Optional in case of self-contained runtimes.
- `stop`
**Description:** Terminate the runtime.
May be triggered by the Requestor on demand or by the Provider Agent due to execution time constraints
specified in the Agreement. Runtime is given a short (< 5s) time window to perform a graceful shutdown, otherwise
it will be forcefully closed.
#### Runtime execution mode
`Runtime::MODE` specifies one of 2 possible execution modes:
- `Server`
The `start` command spawns a background task and returns promptly. Usually, `run` commands have to be invoked
in `Server` mode (see [command execution modes](#command-execution-mode)).
- `Command`
`start` command is a one-shot invocation, which completes promptly. All of the `run` commands are expected to be invoked
via command line.
#### Command execution mode
- `Server`
Command was invoked by a runtime running in a `Server` mode.
Developers may use the `RunCommandExt` trait or `Context::command` to wrap command execution and publish
stdout, stderr and usage counter events in a simple manner.
- `Command`
Command was invoked via command line.
`run_command` implementation should distinguish each of the execution modes but is not required to support both.
### Complementary functions
In addition to the lifecycle and execution logic, runtimes can implement 2 additional customization functions:
- `test`
**Description:** Perform a self-test.
Always executed during Provider Agent startup - i.e. during local yagna provider initialization,
external to the negotiation phase and computation.
Implementation must allow execution at any time.
**Example:** the VM runtime (`ya-runtime-vm`) spawns a Virtual Machine with a minimal test image attached, to verify that
KVM is properly configured on provider's operating system and all bundled components are available in the expected
location on disk.
- `offer`
**Description:** Inject custom properties and / or constraints to an Offer, published by the Provider Agent on the marketplace. `offer`
is executed by the Provider Agent while publishing new offers on the market, also during agent's initialization phase.
Implementation must allow execution at any time.
**Output:** Optional JSON response in the following format:
```json
{
"constraints": "",
"properties": {}
}
```
### Events
(TBD) Future versions of `ya-runtime-sdk` may cover additional types of events:
- runtime state change indication w/ JSON payload
- custom usage counters for billing purposes
---
## Implementation
Runtimes can be implemented by performing the following steps:
- `#[derive(Default, RuntimeDef)]` on a runtime struct
- implement the `Runtime` trait for the struct
- use the `ya_runtime_sdk::run` method to start the runtime
See the [`example-runtime`](examples/example-runtime) for more details.
### Context
Each of the `Runtime` trait functions is parameterized with a mutable reference to the runtime, and a runtime
execution context object.
The `Context` struct exposes the following properties:
- `cli`
Command line arguments provided to the binary
- `conf`
Deserialized runtime configuration
- `conf_path`
Path to the configuration file on a local filesystem
- `env`
An object implementing the `Env` (environment settings) trait
- `emitter`
Command event emitter
`Context` also exposes functions for configuration persistence:
- `read_config`
Read configuration from the specified path. Supports `JSON` (default), `YAML` and `TOML` file formats.
The desired format is
- `write_config`
Write configuration to the specified path.
- `control`
Return a runtime control object. Its `shutdown` method allows to terminate the runtime running in Server mode.
### Configuration
Configuration struct can be set via a `#[conf(..)]` attribute of the `RuntimeDef` derive macro. On runtime startup,
configuration is read from a file located at `~/.local/share//.`.
## Debugging
Developers can use the [ya-runtime-dbg](https://github.com/golemfactory/ya-runtime-dbg) tool to interact with a runtime
running in `Server` mode. See the `README.md` file in the linked repository for more details.
## Deploying
1. Create a `ya-runtime-.json` descriptor file in the **plugins directory**.
```json
[
{
"name": "",
"version": "0.1.0",
"supervisor-path": "exe-unit",
"runtime-path": "/",
"description": "Custom runtime ",
"extra-args": ["--runtime-managed-image"]
}
]
```
2. Edit `~/.local/share/ya-provider/presets.json`:
- create a `presets` entry for `exeunit-name: ""` (e.g. copy an existing preset)
- include the preset name in `active` array
3. Start `golemsp` or `ya-provider`.
The **plugins directory** is by default located at:
- golemsp: `~/.local/lib/yagna/plugins/`
- ya-provider: `/usr/lib/yagna/plugins/`