https://github.com/fgrzl/streams

A basic streaming data store
https://github.com/fgrzl/streams

event-store event-stream streaming streaming-data-pipelines

Last synced: 9 months ago
JSON representation

A basic streaming data store

• Host: GitHub
• URL: https://github.com/fgrzl/streams
• Owner: fgrzl
• License: mit
• Created: 2025-03-04T11:50:40.000Z (10 months ago)
• Default Branch: develop
• Last Pushed: 2025-04-01T19:22:23.000Z (10 months ago)
• Last Synced: 2025-04-01T19:37:10.408Z (10 months ago)
• Topics: event-store, event-stream, streaming, streaming-data-pipelines
• Language: Go
• Homepage:
• Size: 118 KB
• Stars: 0
• Watchers: 0
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE
  • Codeowners: .github/CODEOWNERS

Awesome Lists containing this project

README

          
[![ci](https://github.com/fgrzl/kv/actions/workflows/ci.yml/badge.svg)](https://github.com/fgrzl/kv/actions/workflows/ci.yml)

[![Dependabot Updates](https://github.com/fgrzl/kv/actions/workflows/dependabot/dependabot-updates/badge.svg)](https://github.com/fgrzl/kv/actions/workflows/dependabot/dependabot-updates)

# Streams  

[](url)

#### Organizing Streams: Spaces and Segments  

Streams uses a hierarchical model—**Spaces** and **Segments**—to efficiently manage and consume high-throughput event data.

### Spaces  

A **Space** is a top-level logical container for related streams.

- Groups streams by application, data type, or service  

- Enables broad categorization and easier management  

- Consumers can subscribe to an entire Space for interleaved events from all Segments  

### Segments  

**Segments** are independent, ordered sub-streams within a Space.

- Maintain strict event order  

- Support parallel consumption for scalability  

- Identified uniquely within their Space  

### How It Works  

- **Producing**: Data is written to specific Segments in a Space  

- **Consuming**:  

  - Subscribe to a Space for all Segments (interleaved)  

  - Subscribe to a Segment for strict ordering  

- **Peeking**: Read the latest entry in a Segment without consuming it  

- **Offsets & Transactions**:  

  - Offsets track consumer progress  

  - Transactions ensure consistent writes