https://github.com/proxima-one/indexer-utils-go
https://github.com/proxima-one/indexer-utils-go
indexers library
Last synced: 5 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/proxima-one/indexer-utils-go
- Owner: proxima-one
- Created: 2022-07-27T16:31:18.000Z (almost 4 years ago)
- Default Branch: master
- Last Pushed: 2023-03-15T16:59:13.000Z (over 3 years ago)
- Last Synced: 2024-06-20T10:16:50.343Z (about 2 years ago)
- Topics: indexers, library
- Language: Go
- Homepage:
- Size: 611 KB
- Stars: 0
- Watchers: 5
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: readme.md
Awesome Lists containing this project
README
# Utils for Proxima indexing services
## Logger
To create a Logger you just need to pass `io.Writer` to write to. Typically, it is `os.Stdout`.
```go
logger := utils.NewLogger(os.Stdout)
```
Now you need to register streams. There are two ways to register a stream:
- If your stream has defined start and end points, you can just use:
```go
logger.UpdateStream(streamId, startOffset, endOffset)
```
- Another option is made for live streams:
```go
go logger.StartLiveStreamUpdate(ctx, streamId, startOffset, registry.FindStream, time.Hour)
```
This will update stream metadata according to the passed interval.
You can use `proximaclient.StreamRegistryClient` from [streamdb-client-go](https://github.com/proxima-one/streamdb-client-go) package as a `registry`.
To start logging you need to pass it log interval:
```go
go logger.StartLogging(ctx, 10*time.Second)
```
And pass it every processed event:
```go
logger.EventProcessed(streamId, event)
```
Now it is automatically writing the following tables:
```
╭───────────┬───────────┬─────────────────────┬─────┬───────────┬───────┬───────────┬───────────╮
│ STREAM ID │ HEIGHT │ CURRENT TIMESTAMP │ LAG │ AVG SPEED │ SPEED │ PROCESSED │ REMAINING │
├───────────┼───────────┼─────────────────────┼─────┼───────────┼───────┼───────────┼───────────┤
│ stream.id │ 149977831 │ 2022-12-29 12:15:11 │ 3s │ 2.70 │ 0.70 │ 100.00% │ live │
╰───────────┴───────────┴─────────────────────┴─────┴───────────┴───────┴───────────┴───────────╯
```
Here, the AVG SPEED is an average speed and being calculated as `eventsProcessedSinceStart / timeSinceStart`.
SPEED is "instant speed" - current consumer speed. It is calculated as `eventsSinceLastLog / timeSinceLastLog`.
Every `logger` gorotine stops as its context is closed.