{"id":48436351,"url":"https://github.com/bubustack/bubu-sdk-go","last_synced_at":"2026-04-06T13:02:50.090Z","repository":{"id":317039233,"uuid":"1063254125","full_name":"bubustack/bubu-sdk-go","owner":"bubustack","description":null,"archived":false,"fork":false,"pushed_at":"2025-10-07T13:41:47.000Z","size":89,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-10-15T21:20:08.017Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bubustack.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-09-24T11:27:14.000Z","updated_at":"2025-10-13T14:01:02.000Z","dependencies_parsed_at":"2025-09-28T13:35:59.484Z","dependency_job_id":null,"html_url":"https://github.com/bubustack/bubu-sdk-go","commit_stats":null,"previous_names":["bubustack/bubu-sdk-go"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/bubustack/bubu-sdk-go","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bubustack%2Fbubu-sdk-go","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bubustack%2Fbubu-sdk-go/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bubustack%2Fbubu-sdk-go/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bubustack%2Fbubu-sdk-go/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bubustack","download_url":"https://codeload.github.com/bubustack/bubu-sdk-go/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bubustack%2Fbubu-sdk-go/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31473273,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-06T08:36:52.050Z","status":"ssl_error","status_checked_at":"2026-04-06T08:36:51.267Z","response_time":112,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-04-06T13:02:49.402Z","updated_at":"2026-04-06T13:02:50.076Z","avatar_url":"https://github.com/bubustack.png","language":"Go","readme":"# 🧰 bubu-sdk-go — Official Go SDK for bobrapet\n\n[![Go Reference](https://pkg.go.dev/badge/github.com/bubustack/bubu-sdk-go.svg)](https://pkg.go.dev/github.com/bubustack/bubu-sdk-go)\n[![Go Report Card](https://goreportcard.com/badge/github.com/bubustack/bubu-sdk-go)](https://goreportcard.com/report/github.com/bubustack/bubu-sdk-go)\n\nThe official Go SDK for building type-safe, production-grade components for **bobrapet**, the Kubernetes‑native AI and data workflow orchestration engine.\n\nQuick links:\n- SDK docs: https://bubustack.io/docs/sdk\n- API Reference: https://pkg.go.dev/github.com/bubustack/bubu-sdk-go\n\n## 🌟 Key Features\n\nUse this SDK to build **engrams** (data processing tasks) and **impulses** (event listeners that trigger workflows). bobrapet orchestrates their execution as Kubernetes Jobs and Deployments, handling:\n\n- **Type-safe configuration and inputs** — Define your interfaces as Go structs, get compile-time safety.\n- **Automatic large payload handling** — Outputs exceeding size limits are transparently offloaded to S3/file storage.\n- **Streaming pipelines** — Build real-time data processing chains with gRPC bidirectional streaming.\n- **Retries and observability** — Exit codes inform retry policies; OpenTelemetry metrics/tracing hooks included (initialize an exporter in your app/infra).\n\n## 🏗️ Architecture\n\nHigh-level SDK architecture, execution modes, and operator integration are documented here:\n- Overview: https://bubustack.io/docs/sdk\n\n### 🧭 When to use which mode\n\n- Batch (Jobs): finite tasks with clear start/end; use `StartBatch`. Evidence: batch flow in `batch.go`.\n- Streaming (Deployments with gRPC): continuous processing with backpressure/heartbeats; use `StartStreaming`. Evidence: `stream.go`.\n- Impulse (Deployments): long‑running trigger that creates `StoryRun`s; use `RunImpulse`. Evidence: `impulse.go`.\n\n---\n\n## 🚀 Quick Start\n\nLet's build a simple batch engram that greets users.\n\n### 1. Create your Go module\n\n```bash\nmkdir hello-engram \u0026\u0026 cd hello-engram\ngo mod init github.com/yourusername/hello-engram\ngo get github.com/bubustack/bubu-sdk-go@latest\n```\n\n### 2. Write the code\n\nCreate a single `main.go` file with all the necessary components.\n\n```go\npackage main\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\t\"log\"\n\n\tsdk \"github.com/bubustack/bubu-sdk-go\"\n\t\"github.com/bubustack/bubu-sdk-go/engram\"\n)\n\n// Config holds static configuration from the Engram resource 'with' block.\ntype Config struct {\n\tDefaultGreeting string `mapstructure:\"defaultGreeting\"`\n}\n\n// Inputs holds runtime data passed to this execution via StepRun inputs.\ntype Inputs struct {\n\tName string `mapstructure:\"name\"`\n}\n\n// GreeterEngram implements the engram.Batch interface.\ntype GreeterEngram struct {\n\tgreeting string\n}\n\n// NewGreeter creates a new GreeterEngram.\nfunc NewGreeter() *GreeterEngram {\n\treturn \u0026GreeterEngram{}\n}\n\n// Init is called once when the engram starts.\nfunc (g *GreeterEngram) Init(ctx context.Context, cfg Config, secrets *engram.Secrets) error {\n\tg.greeting = \"Hello\"\n\tif cfg.DefaultGreeting != \"\" {\n\t\tg.greeting = cfg.DefaultGreeting\n\t}\n\treturn nil\n}\n\n// Process is the core logic. It receives typed inputs and returns a Result.\nfunc (g *GreeterEngram) Process(ctx context.Context, ec *engram.ExecutionContext, inputs Inputs) (*engram.Result, error) {\n\tif inputs.Name == \"\" {\n\t\treturn nil, fmt.Errorf(\"input 'name' is required\")\n\t}\n\tmessage := fmt.Sprintf(\"%s, %s!\", g.greeting, inputs.Name)\n\treturn engram.NewResultFrom(map[string]any{\n\t\t\"greeting\": message,\n\t})\n}\n\nfunc main() {\n\tif err := sdk.StartBatch(context.Background(), NewGreeter()); err != nil {\n\t\tlog.Fatalf(\"Engram failed: %v\", err)\n\t}\n}\n```\n\n### 3. Build the binary\n\n```bash\ngo build -o hello-engram .\n```\n\n### 4. Containerize and deploy\n\nCreate a `Dockerfile`:\n\n```dockerfile\n# Use a smaller base image\nFROM golang:1.24-alpine AS builder\nWORKDIR /app\nCOPY go.mod go.sum ./\nRUN go mod download\nCOPY . .\n# Build a static binary\nRUN CGO_ENABLED=0 go build -o /hello-engram .\n\n# Final stage\nFROM alpine:latest\n# Add non-root user\nRUN addgroup -S app \u0026\u0026 adduser -S app -G app\nUSER app\n# Copy binary and certificates\nCOPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/\nCOPY --from=builder /hello-engram /hello-engram\nENTRYPOINT [\"/hello-engram\"]\n```\n\nBuild and push the image to your container registry:\n\n```bash\ndocker build -t myregistry.io/hello-engram:latest .\ndocker push myregistry.io/hello-engram:latest\n```\n\n### 5. Deploy to Kubernetes\n\nCreate an `Engram` resource (`hello-engram.yaml`):\n\n```yaml\napiVersion: bubustack.io/v1alpha1\nkind: Engram\nmetadata:\n  name: hello-engram\nspec:\n  image: myregistry.io/hello-engram:latest\n  with:\n    defaultGreeting: \"Greetings\"\n```\n\nCreate a `Story` that uses the engram (`greet-story.yaml`):\n\n```yaml\napiVersion: bubustack.io/v1alpha1\nkind: Story\nmetadata:\n  name: greet-users\nspec:\n  steps:\n    - name: greet\n      engram: hello-engram\n      with:\n        name: \"{{ .inputs.userName }}\"\n```\n\nApply the resources and trigger a `StoryRun`:\n\n```bash\nkubectl apply -f hello-engram.yaml -f greet-story.yaml\n\nkubectl create -f - \u003c\u003cEOF\napiVersion: bubustack.io/v1alpha1\nkind: StoryRun\nmetadata:\n  generateName: greet-users-\nspec:\n  storyRef:\n    name: greet-users\n  inputs:\n    userName: \"Bob\"\nEOF\n```\n\n---\n\n## 📚 Core concepts\n\n### Engrams\n\n**Engrams** are the building blocks of workflows. They are stateless, single-purpose components that execute a specific task.\n\n| Type | Use Case | Kubernetes Workload |\n|------|----------|---------------------|\n| **BatchEngram** | Data transformations, API calls, ETL tasks | Job |\n| **StreamingEngram** | Real-time data processing, filtering, routing | Deployment (gRPC server) |\n\n### Impulses\n\n**Impulses** are long-running services that listen for external events (webhooks, message queues, schedulers) and trigger StoryRuns.\n\n| Component | Role | Kubernetes Workload |\n|-----------|------|---------------------|\n| **Impulse** | Event listener → Story trigger | Deployment |\n\n---\n\n## 🌟 Key features\n\n### Type-Safe and Generic\n\nDefine your configuration and inputs as native Go structs. The SDK handles deserialization and validation:\n\n```go\ntype Config struct {\n    APIKey string `mapstructure:\"apiKey\"`\n    Timeout time.Duration `mapstructure:\"timeout\"` // Supports duration parsing\n}\n```\n\n### Transparent Storage Offloading\n\nLarge outputs are automatically offloaded to S3 or file storage, keeping Kubernetes resources lean:\n\n```go\nreturn \u0026sdk.Result{\n    Data: map[string]any{\n        \"largePayload\": someLargeData, // Automatically offloaded if \u003e BUBU_MAX_INLINE_SIZE\n    },\n}, nil\n```\n\n### Streaming Pipelines\n\nBuild real-time data processing chains with metadata propagation for tracing:\n\n```go\nfunc (s *Streamer) Stream(ctx context.Context, in \u003c-chan engram.StreamMessage, out chan\u003c- engram.StreamMessage) error {\n    for msg := range in {\n        // Process msg.Payload, propagate msg.Metadata for tracing\n        out \u003c- engram.StreamMessage{\n            Metadata: msg.Metadata, // Preserve trace IDs\n            Payload: processedData,\n        }\n    }\n    return nil\n}\n```\n\n### Retries and Exit Codes\n\nThe SDK patches StepRun status with exit codes that inform the operator's retry policy:\n\n- **0**: Success\n- **1**: Logic error (terminal, no retry)\n- **124**: Timeout (retryable)\n\n---\n\n## 📚 Documentation\n\n- SDK: https://bubustack.io/docs/sdk\n- Guides: https://bubustack.io/docs\n- Reference: https://bubustack.io/docs/reference\n- How-to: https://bubustack.io/docs/howto\n- Troubleshooting: https://bubustack.io/docs/troubleshooting\n\n---\n\n## ✅ Support matrix\n\n| Component | Version |\n|-----------|---------|\n| **Go** | 1.24+ |\n| **Kubernetes** | 1.28+ (bobrapet operator compatibility) |\n| **bobrapet operator** | v0.1.0+ |\n\n---\n\n## ⚙️ Environment variables\n\nThe SDK is controlled entirely by environment variables injected by the bobrapet operator. See [docs/reference/config](https://bubustack.io/docs/reference/config) for the complete reference.\n\nKey variables:\n\n- `BUBU_STEP_TIMEOUT` — Batch execution timeout (default: 30m)\n- `BUBU_STORAGE_PROVIDER` — Storage backend: `s3`, `file`, or unset (disabled)\n- `BUBU_MAX_INLINE_SIZE` — Offload threshold in bytes (default: 1024)\n- `BUBU_GRPC_PORT` — gRPC server port for streaming engrams (default: 50051)\n- `BUBU_EXECUTION_MODE` — Set by operator: `batch` | `streaming` (evidence in controllers)\n- `BUBU_STORAGE_PATH` — Required when `BUBU_STORAGE_PROVIDER=file`; base directory for file store\n- `BUBU_MAX_RECURSION_DEPTH` — Max traversal depth for hydrate/dehydrate (default: 10)\n- `BUBU_STORAGE_TIMEOUT` — Timeout for storage operations (default: 5m)\n\n---\n\n## 🛠️ Local Development\n\n1.  **Clone the repository:**\n    ```bash\n    git clone https://github.com/bubustack/bubu-sdk-go.git\n    cd bubu-sdk-go\n    ```\n\n2.  **Run tests:**\n    ```bash\n    make test\n    ```\n\n3.  **Lint:**\n    ```bash\n    make lint\n    ```\n\n4.  **Run all checks:**\n    ```bash\n    make all\n    ```\n\n---\n\n## 📢 Support, Security, and Changelog\n\n- See `SUPPORT.md` for how to get help and report issues.\n- See `SECURITY.md` for vulnerability reporting and security posture.\n- See `CHANGELOG.md` for version history.\n\n## 🤝 Community\n\n- Code of Conduct: see [CODE_OF_CONDUCT.md](./CODE_OF_CONDUCT.md) (Contributor Covenant v3.0)\n\n## 📄 License\n\nCopyright 2025 BubuStack.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbubustack%2Fbubu-sdk-go","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbubustack%2Fbubu-sdk-go","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbubustack%2Fbubu-sdk-go/lists"}