Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/liftbridge-io/liftbridge-api
Protobuf definitions for the Liftbridge gRPC API. https://github.com/liftbridge-io/liftbridge
https://github.com/liftbridge-io/liftbridge-api
api client cloud-native grpc liftbridge messaging nats protobuf pubsub streaming
Last synced: 2 months ago
JSON representation
Protobuf definitions for the Liftbridge gRPC API. https://github.com/liftbridge-io/liftbridge
- Host: GitHub
- URL: https://github.com/liftbridge-io/liftbridge-api
- Owner: liftbridge-io
- License: apache-2.0
- Created: 2018-06-29T01:36:48.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2022-11-23T15:10:47.000Z (about 2 years ago)
- Last Synced: 2024-10-01T15:29:40.111Z (3 months ago)
- Topics: api, client, cloud-native, grpc, liftbridge, messaging, nats, protobuf, pubsub, streaming
- Language: Python
- Homepage:
- Size: 824 KB
- Stars: 15
- Watchers: 4
- Forks: 13
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Liftbridge API
This repository contains the public API definitions for
[Liftbridge](https://github.com/liftbridge-io/liftbridge). It is primarily
intended for Liftbridge client developers.
## gRPC API
The client-facing gRPC API is defined in [api.proto](api.proto).
## Direct NATS API
It is also possible for a client to publish messages to Liftbridge via NATS
directly. Liftbridge accepts plain NATS messages, allowing it to make existing
subjects durable without any publisher changes. However, these messages will
not have some features such as acks.
In order to opt into Liftbridge-specific features, the message must be prefixed
with the following header and be encoded as a `Message` (defined in
[api.proto](api.proto)).
### Liftbridge Envelope Header
All Liftbridge messages and RPCs sent over NATS are prefixed with an envelope
header. This includes client-facing messages, such as publishes and acks, as
well as internal RPCs like replication.
```
0 8 16 24 32
├───────────────┴───────────────┴───────────────┴───────────────┤
│ Magic Number │
├───────────────┬───────────────┬───────────────┬───────────────┤
│ Version │ HeaderLen │ Flags │ MsgType │
├───────────────┴───────────────┴───────────────┴───────────────┤
│ CRC-32C (optional) │
└───────────────────────────────────────────────────────────────┘
```
#### Magic number [4 bytes]
The Liftbridge magic number is `B9 0E 43 B4`. This was chosen by random but
deliberately restricted to invalid UTF-8 to reduce the chance of a collision.
This was also verified to not match known file signatures.
#### Version [1 byte]
The version byte allows for future protocol upgrades. This should only be
bumped if the envelope format changes or if the message encoding changes in a
non-backwards-compatible way. Adding fields to the messages should not require
a version bump.
Currently, the only supported protocol version is v0, i.e. `0x00`.
#### HeaderLen [1 byte]
The header length is the offset of the payload. This is included primarily for
safety.
#### Flags [1 byte]
The flag bits are defined as follows:
| Bit | Description |
| :-- | :-------------- |
| 0 | CRC-32C enabled |
#### MsgType [1 byte]
This is the Liftbridge-specific message type the envelope contains:
| MsgType | Name | Description | Internal |
| :------ | :------------------------ | :----------------------------------------------------- | :------- |
| 0 | Publish | Client-published message | no |
| 1 | Ack | Server-published ack | no |
| 2 | ReplicationRequest | Request to replicate partition data | yes |
| 3 | ReplicationResponse | Response to ReplicationRequest | yes |
| 4 | RaftJoinRequest | Request to join Raft cluster | yes |
| 5 | RaftJoinResponse | Response to RaftJoinRequest | yes |
| 6 | LeaderEpochOffsetRequest | Request for partition leader's latest offset for epoch | yes |
| 7 | LeaderEpochOffsetResponse | Response to LeaderEpochOffsetRequest | yes |
| 8 | PropagatedRequest | Request forwarded to metadata leader | yes |
| 9 | PropagatedResponse | Response to PropagatedRequest | yes |
| 10 | ServerInfoRequest | Request for cluster information | yes |
| 11 | ServerInfoResponse | Response to ServerInfoRequest | yes |
| 12 | PartitionStatusRequest | Request to get partition status | yes |
| 13 | PartitionStatusResponse | Response to PartitionStatusRequest | yes |
| 14 | PartitionNotification | Signal new data is available for partition | yes |
#### CRC-32C [4 bytes, optional]
The CRC-32C (Castagnoli) is the checksum of the payload (i.e. from HeaderLen to
the end). This is optional but should significantly reduce the chance that a
random NATS message is interpreted as a Liftbridge message.