{"id":35600555,"url":"https://github.com/tiny-systems/module","last_synced_at":"2026-05-09T00:02:07.814Z","repository":{"id":157649266,"uuid":"632659874","full_name":"tiny-systems/module","owner":"tiny-systems","description":"All-in-one library to run, test and build Tiny Systems modules.","archived":false,"fork":false,"pushed_at":"2026-05-07T19:19:50.000Z","size":1261,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-07T21:19:15.901Z","etag":null,"topics":["automation","crd","flow-based-programming","golang","kubebuilder","kubernetes","kubernetes-operator","sdk","visual-programming","workflow-engine"],"latest_commit_sha":null,"homepage":"https://docs.tinysystems.io","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tiny-systems.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-04-25T21:43:45.000Z","updated_at":"2026-05-07T19:19:52.000Z","dependencies_parsed_at":null,"dependency_job_id":"eb3e98e2-92b7-4d7e-837f-c039fab092e9","html_url":"https://github.com/tiny-systems/module","commit_stats":null,"previous_names":[],"tags_count":499,"template":false,"template_full_name":null,"purl":"pkg:github/tiny-systems/module","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tiny-systems%2Fmodule","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tiny-systems%2Fmodule/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tiny-systems%2Fmodule/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tiny-systems%2Fmodule/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tiny-systems","download_url":"https://codeload.github.com/tiny-systems/module/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tiny-systems%2Fmodule/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32802533,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-08T08:22:46.396Z","status":"ssl_error","status_checked_at":"2026-05-08T08:22:45.650Z","response_time":54,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automation","crd","flow-based-programming","golang","kubebuilder","kubernetes","kubernetes-operator","sdk","visual-programming","workflow-engine"],"created_at":"2026-01-05T01:21:12.561Z","updated_at":"2026-05-09T00:02:07.793Z","avatar_url":"https://github.com/tiny-systems.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Tiny Systems Module SDK\n\nA Kubebuilder-based Kubernetes operator SDK for building flow-based workflow engines. This SDK provides the complete infrastructure for developing modular operators that can be composed into visual workflows.\n\n## Overview\n\nTiny Systems Module SDK enables developers to create **module operators** (like `common-module`, `http-module`, `grpc-module`) that bring specific functionality into a Kubernetes-native flow engine. Each module provides reusable components that can be connected through a port-based architecture to create complex workflows.\n\n### Key Features\n\n- **Port-Based Component Architecture**: Visual programming model with input/output ports\n- **JSON Schema-Driven Configuration**: Automatic schema generation with UI hints\n- **Message Routing \u0026 Retry Logic**: Intelligent routing with exponential backoff\n- **Multi-Module Communication**: gRPC-based inter-module communication\n- **Expression-Based Data Transformation**: Mustache-style `{{expression}}` syntax with JSONPath for flexible data mapping\n- **Kubernetes-Native**: Everything is a CRD with standard controller patterns\n- **OpenTelemetry Integration**: Built-in observability with tracing and metrics\n- **CLI Tools**: Complete tooling for running and building modules\n\n## Table of Contents\n\n- [Overview](#overview)\n- [Architecture](#architecture)\n  - [Core Concepts](#core-concepts)\n  - [SDK Components](#sdk-components)\n  - [Communication Patterns](#communication-patterns)\n  - [Design Patterns](#design-patterns)\n  - [Scalability](#scalability)\n  - [Project Structure](#project-structure)\n- [Getting Started](#getting-started)\n- [Helm Charts](#helm-charts)\n- [Development](#development)\n  - [Running on the Cluster](#running-on-the-cluster)\n  - [Testing](#test-it-out)\n  - [Modifying the API](#modifying-the-api-definitions)\n- [Module Development](#module-development)\n  - [Quick Start](#quick-start)\n  - [Component Interface](#component-interface-deep-dive)\n  - [Configuration Schemas](#configuration-schemas)\n  - [System Ports](#system-ports)\n  - [Error Handling](#error-handling)\n  - [Resource Manager](#using-the-resource-manager)\n  - [Observability](#observability)\n  - [Testing](#testing-components)\n  - [Best Practices](#best-practices)\n- [Contributing](#contributing)\n- [License](#license)\n\n## Architecture\n\n### Core Concepts\n\nThe SDK is built around several key abstractions:\n\n#### Custom Resource Definitions (CRDs)\n\n- **TinyNode**: Core execution unit representing a component instance in a flow\n  - Contains module name, version, component type, port configurations, and edges\n  - Status includes module/component info, port schemas, and error state\n\n- **TinyModule**: Registry of available modules for service discovery\n  - References container image\n  - Status contains module address (gRPC), version, and list of components\n\n- **TinyFlow**: Logical grouping of nodes representing a workflow\n  - Uses labels to associate nodes together\n\n- **TinyProject**: Top-level organizational unit grouping multiple flows\n\n- **TinySignal**: External trigger mechanism for node execution\n  - Specifies target node, port, and data payload\n  - One-off: deleted immediately after successful delivery\n  - Works like webhooks or manual triggers\n\n- **TinyTracker**: Execution monitoring for detailed tracing\n\n- **TinyWidgetPage**: Custom UI dashboard pages for visualization\n\n#### Component System\n\nComponents are the building blocks of workflows. Each component:\n\n- Implements the `Component` interface (`module/component.go`)\n- Defines **ports** for input/output with positions (Top/Right/Bottom/Left)\n- Handles messages through the `Handle()` method\n- Provides JSON Schema configuration for each port\n\n#### Message Flow\n\n**Standard message flow:**\n```\nExternal Trigger (TinySignal)\n  → TinySignal Controller\n    → Scheduler.Handle()\n      → Runner.MsgHandler()\n        → Component.Handle()\n          → output() callback\n            → Next node via edges\n  → TinySignal deleted (one-off)\n```\n\n#### Port System\n\nPorts enable component communication:\n- **Source Ports**: Output ports that send data\n- **Target Ports**: Input ports that receive data\n- **System Ports**:\n  - `_reconcile`: Triggers node reconciliation\n  - `_client`: Receives Kubernetes client for resource operations\n  - `_settings`: Configuration port\n  - `_control`: Dashboard control port\n  - `_identity`: Receives node identity (name, namespace, flow, project) for resource namespacing\n\nEach port can have:\n- **Configuration**: JSON schema defining expected input structure\n- **Response Configuration**: Schema for output data structure\n\n#### Edge Configuration\n\nEdges connect ports between nodes and support data transformation using mustache-style expressions:\n\n**Expression Syntax**: Use `{{expression}}` to evaluate JSONPath expressions against incoming data.\n\n```json\n{\n  \"body\": \"{{$.request.body}}\",\n  \"statusCode\": 200,\n  \"greeting\": \"Hello {{$.user.name}}!\",\n  \"isAdmin\": \"{{$.user.role == 'admin'}}\"\n}\n```\n\n**Expression Types**:\n- **Pure expression** (`\"{{$.field}}\"`) - Returns the actual type (string, number, boolean, object)\n- **Literal value** (`\"hello\"`, `200`, `true`) - Static values passed through as-is\n- **String interpolation** (`\"Hello {{$.name}}!\"`) - Embeds expression results in strings\n- **JSONPath with operators** (`\"{{$.count + 1}}\"`, `\"{{$.method == 'GET'}}\"`) - Supports arithmetic and comparison\n\n**Key Features**:\n- Type preservation: `\"{{$.count}}\"` returns a number, not a string\n- Graceful error handling: If source data is unavailable, expressions return `nil`\n- Full JSONPath support via [ajson](https://github.com/tiny-systems/ajson) library\n\n**Built-in Functions**:\n\nThe expression engine includes many built-in functions:\n\n| Category | Functions |\n|----------|-----------|\n| **Array** | `length()`, `first()`, `last()`, `avg()`, `sum()`, `size()` |\n| **String** | `upper()`, `lower()`, `trim()`, `reverse()`, `b64encode()`, `b64decode()` |\n| **String (multi-arg)** | `split(str, sep)`, `join(arr, sep)`, `contains(str, sub)`, `hasprefix(str, prefix)`, `hassuffix(str, suffix)`, `replace(str, old, new)`, `substr(str, start[, len])`, `index(str, sub)` |\n| **Math** | `abs()`, `ceil()`, `floor()`, `round()`, `sqrt()`, `pow10()` |\n| **Logic** | `not()`, ternary `? :` |\n\n**String Function Examples**:\n```json\n{\n  \"kind\": \"{{first(split($.target, '/'))}}\",\n  \"name\": \"{{last(split($.target, '/'))}}\",\n  \"hasNamespace\": \"{{contains($.args, '-n ')}}\",\n  \"upperName\": \"{{upper($.name)}}\",\n  \"label\": \"{{replace($.label, '=', ': ')}}\",\n  \"domain\": \"{{first(split(last(split($.url, '//')), '/'))}}\",\n  \"message\": \"{{hasprefix($.error, 'NotFound') ? 'Resource not found' : $.error}}\"\n}\n```\n\n### SDK Components\n\nThe SDK provides several packages for module developers:\n\n#### `/module/` - Core Interfaces\n- `Component`: Main interface for component implementation\n- `Port`: Port definitions and configuration\n- `Handler`: Callback function type for message routing\n\n#### `/pkg/resource/` - Resource Manager\nUnified Kubernetes client providing:\n- Node/Module/Flow/Project CRUD operations\n- Signal creation for triggering nodes\n- Ingress/Service management for exposing ports\n- Helm release management\n\n#### `/pkg/schema/` - JSON Schema Generator\n- Automatic schema generation from Go structs\n- Custom struct tags for UI hints: `configurable`, `shared`, `propertyOrder`, `tab`, `align`\n- Definition sharing and override mechanism\n\n#### `/pkg/evaluator/` - Expression Evaluator\n- Mustache-style `{{expression}}` syntax processing\n- JSONPath evaluation via ajson library\n- Type-preserving evaluation (numbers stay numbers, booleans stay booleans)\n- Graceful error handling when source data is unavailable\n\n#### `/pkg/metrics/` - Observability\n- OpenTelemetry integration with spans\n- Metrics (gauges, counters)\n- Tracker system for flow execution monitoring\n\n#### `/internal/scheduler/` - Message Routing\n- Manages component instances\n- Routes messages between nodes\n- Handles cross-module communication via gRPC\n\n#### `/internal/client/` - Client Pool\n- Manages gRPC connections to other modules\n- Connection pooling and lifecycle management\n\n#### `/cli/` - Command-Line Tools\n- `run`: Complete operator runtime\n- `build`: Module building and publishing\n\n### Communication Patterns\n\n#### Same-Module Communication\nWhen nodes belong to the same module, messages are routed directly through the scheduler for optimal performance.\n\n#### Cross-Module Communication\nWhen nodes belong to different modules:\n1. Scheduler identifies the target module via TinyModule CRD\n2. Client pool establishes/reuses gRPC connection\n3. Message is sent to target module's gRPC server\n4. Target module's scheduler routes to the appropriate component\n\n#### Retry Mechanism\n- **Transient Errors**: Exponential backoff (1s → 30s max)\n- **Permanent Errors**: Marked via `PermanentError` wrapper to stop retries\n- Context cancellation stops all retries\n\n### Design Patterns\n\n1. **Eventual Consistency with Reconciliation**: Periodic reconciliation (every 5 minutes) plus signal-based immediate updates\n2. **Leader Election**: Leader handles control operations and blocking handlers for multi-pod coordination\n3. **Schema-Driven Configuration**: Go structs automatically generate JSON schemas for UI integration\n4. **Expression-Based Transformation**: Mustache-style `{{expression}}` syntax with JSONPath enables flexible data mapping without code changes\n5. **Definition Sharing**: Components mark fields as `shared:true` or `configurable:true` for cross-node type safety\n6. **One-off Signals**: TinySignals are deleted after delivery for clean trigger semantics\n\n### Scalability\n\nThe SDK supports horizontal scaling of module operators with leader election for coordination.\n\n#### Leader Election\n\n- Uses Kubernetes native leader election via `k8s.io/client-go/tools/leaderelection` with Lease resources\n- Configuration: 15s lease duration, 10s renew deadline, 2s retry period\n- Each pod identifies itself using the `HOSTNAME` environment variable\n- Leadership state tracked via `isLeader` atomic boolean\n- **Purpose**: Only leader runs blocking handlers (e.g., HTTP servers, long-running processes)\n\n#### One-off TinySignals\n\n- TinySignals are deleted immediately after successful delivery\n- Clean trigger semantics without signal accumulation\n\n### Project Structure\n\n```\n.\n├── api/v1alpha1/          # CRD definitions (TinyNode, TinyModule, TinyFlow, etc.)\n├── module/                # Core SDK interfaces for component developers\n│   ├── component.go       # Component interface\n│   ├── node.go           # Port definitions\n│   └── handler.go        # Handler function type\n├── pkg/                   # Reusable SDK packages\n│   ├── resource/         # Kubernetes resource manager\n│   ├── schema/           # JSON schema generator\n│   ├── evaluator/        # JSONPath expression evaluator\n│   ├── errors/           # Error handling utilities\n│   └── metrics/          # OpenTelemetry integration\n├── internal/              # Internal operator implementation\n│   ├── controller/       # Kubernetes controllers\n│   ├── scheduler/        # Message routing and execution\n│   ├── server/           # gRPC server\n│   └── client/           # gRPC client pool\n├── cli/                   # Command-line tools (run, build)\n├── registry/              # Component registration system\n├── config/                # Kubernetes manifests and CRD definitions\n│   ├── crd/              # CRD YAML files\n│   ├── samples/          # Example resources\n│   └── rbac/             # RBAC configurations\n└── charts/                # Helm charts for deployment\n```\n\n## Getting Started\nYou’ll need a Kubernetes cluster to run against. You can use [KIND](https://sigs.k8s.io/kind) to get a local cluster for testing, or run against a remote cluster.\n**Note:** Your controller will automatically use the current context in your kubeconfig file (i.e. whatever cluster `kubectl cluster-info` shows).\n\n## Helm charts\n```shell\nhelm repo add tinysystems https://tiny-systems.github.io/module/\nhelm repo update # if you already added repo before\nhelm install my-corp-data-processing-tools --set controllerManager.manager.image.repository=registry.mycorp/tools/data-processing  tinysystems/tinysystems-operator\n```\n\n### Running on the cluster\n1. Install Instances of Custom Resources:\n\n```shell\nkubectl apply -f config/samples/\n```\n\n2. Build and push your image to the location specified by `IMG`:\n\n```shell\nmake docker-build docker-push IMG=\u003csome-registry\u003e/operator:tag\n```\n\n3. Deploy the controller to the cluster with the image specified by `IMG`:\n\n```shell\nmake deploy IMG=\u003csome-registry\u003e/operator:tag\n```\n\n\n### Undeploy controller\nUnDeploy the controller from the cluster:\n\n```shell\nmake undeploy\n```\n\n## Contributing\n// TODO(user): Add detailed information on how you would like others to contribute to this project\n\n### How it works\nThis project aims to follow the Kubernetes [Operator pattern](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/).\n\nIt uses [Controllers](https://kubernetes.io/docs/concepts/architecture/controller/),\nwhich provide a reconcile function responsible for synchronizing resources until the desired state is reached on the cluster.\n\n### Test It Out\n1. Install the CRDs into the cluster:\n\n```shell\nmake install\n```\n\n2. Run your controller (this will run in the foreground, so switch to a new terminal if you want to leave it running):\n\n```shell\nmake run\n```\n\n**NOTE:** You can also run this in one step by running: `make install run`\n\n### Modifying the API definitions\nIf you are editing the API definitions, generate the manifests such as CRs or CRDs using:\n\n```shell\nmake manifests\n```\n\n### Create new api\n```shell\nkubebuilder create api --group operator --version v1alpha1 --kind TinySignal\n```\n\n**NOTE:** Run `make --help` for more information on all potential `make` targets\n\nMore information can be found via the [Kubebuilder Documentation](https://book.kubebuilder.io/introduction.html)\n\n### Module Development\n\nThis SDK provides everything you need to build custom module operators. Here's a complete guide to developing your own module.\n\n#### Quick Start\n\n1. **Create a New Go Project**\n\n```bash\nmkdir my-module\ncd my-module\ngo mod init github.com/myorg/my-module\n```\n\n2. **Add SDK Dependency**\n\n```bash\ngo get github.com/tiny-systems/module\n```\n\n3. **Implement a Component**\n\nCreate `components/hello.go`:\n\n```go\npackage components\n\nimport (\n    \"context\"\n    \"github.com/tiny-systems/module/module\"\n)\n\ntype Hello struct{}\n\n// Configuration for the input port\ntype HelloInput struct {\n    Name string `json:\"name\" configurable:\"true\"`\n}\n\n// Configuration for the output\ntype HelloOutput struct {\n    Greeting string `json:\"greeting\" shared:\"true\"`\n}\n\nfunc (h *Hello) Instance() module.Component {\n    return \u0026Hello{}\n}\n\nfunc (h *Hello) GetInfo() module.ComponentInfo {\n    return module.ComponentInfo{\n        Name:        \"hello\",\n        Description: \"Greets a person by name\",\n        Info:        \"Simple greeting component example\",\n        Tags:        []string{\"example\", \"greeting\"},\n    }\n}\n\nfunc (h *Hello) Ports() []module.Port {\n    return []module.Port{\n        {\n            Name:          \"input\",\n            Label:         \"Input\",\n            Source:        false, // This is an input port\n            Position:      module.Left,\n            Configuration: \u0026HelloInput{},\n        },\n        {\n            Name:     \"output\",\n            Label:    \"Output\",\n            Source:   true, // This is an output port\n            Position: module.Right,\n            Configuration: \u0026HelloOutput{},\n        },\n        {\n            Name:     \"error\",\n            Label:    \"Error\",\n            Source:   true,\n            Position: module.Bottom,\n        },\n    }\n}\n\nfunc (h *Hello) Handle(ctx context.Context, output module.Handler, port string, message any) any {\n    if port == \"input\" {\n        // Parse input configuration\n        input := message.(*HelloInput)\n\n        // Create greeting\n        greeting := \"Hello, \" + input.Name + \"!\"\n\n        // Send to output port\n        output(ctx, \"output\", \u0026HelloOutput{\n            Greeting: greeting,\n        })\n    }\n    return nil\n}\n```\n\n4. **Register Component and Create Main**\n\nCreate `main.go`:\n\n```go\npackage main\n\nimport (\n    \"github.com/myorg/my-module/components\"\n    \"github.com/tiny-systems/module/cli\"\n    \"github.com/tiny-systems/module/registry\"\n)\n\nfunc main() {\n    // Register all components\n    registry.Register(\u0026components.Hello{})\n\n    // Run the operator\n    cli.Run()\n}\n```\n\n5. **Run Your Module**\n\n```bash\n# Build\ngo build -o my-module\n\n# Run locally (connects to your current kubectl context)\n./my-module run --name=my-module --version=1.0.0 --namespace=tinysystems\n```\n\n6. **Deploy to Kubernetes**\n\n```bash\n# Build and push Docker image\ndocker build -t myregistry/my-module:1.0.0 .\ndocker push myregistry/my-module:1.0.0\n\n# Install using Helm\nhelm repo add tinysystems https://tiny-systems.github.io/module/\nhelm install my-module \\\n  --set controllerManager.manager.image.repository=myregistry/my-module \\\n  --set controllerManager.manager.image.tag=1.0.0 \\\n  tinysystems/tinysystems-operator\n```\n\n#### Component Interface Deep Dive\n\n##### GetInfo() - Component Metadata\n\n```go\nfunc (c *MyComponent) GetInfo() module.ComponentInfo {\n    return module.ComponentInfo{\n        Name:        \"my-component\",      // Unique identifier\n        Description: \"Does something\",     // Short description\n        Info:        \"Detailed info...\",   // Long description\n        Tags:        []string{\"tag1\"},     // Searchable tags\n    }\n}\n```\n\n##### Ports() - Define Component Ports\n\nPorts define how components connect to each other:\n\n```go\nfunc (c *MyComponent) Ports() []module.Port {\n    return []module.Port{\n        {\n            Name:          \"input\",           // Unique port name\n            Label:         \"Input Data\",      // Display label\n            Source:        false,             // Input port\n            Position:      module.Left,       // Visual position\n            Configuration: \u0026InputConfig{},    // Expected data structure\n        },\n        {\n            Name:              \"output\",\n            Label:             \"Output Data\",\n            Source:            true,          // Output port\n            Position:          module.Right,\n            Configuration:     \u0026OutputConfig{}, // Output data structure\n        },\n    }\n}\n```\n\n**Port Positions**: `module.Top`, `module.Right`, `module.Bottom`, `module.Left`\n\n##### Handle() - Process Messages\n\nThe `Handle` method is called when a message arrives on a port:\n\n```go\nfunc (c *MyComponent) Handle(\n    ctx context.Context,\n    output module.Handler,\n    port string,\n    message any,\n) any {\n    switch port {\n    case \"input\":\n        // Type assert the message\n        input := message.(*InputConfig)\n\n        // Do work\n        result := processData(input)\n\n        // Send to output port\n        output(ctx, \"output\", \u0026OutputConfig{\n            Result: result,\n        })\n\n    case \"_reconcile\":\n        // Handle reconciliation (called periodically)\n        // Use this for cleanup, state sync, etc.\n\n    case v1alpha1.IdentityPort:\n        // Receive node identity for resource namespacing\n        id := message.(v1alpha1.NodeIdentity)\n        c.storagePath = filepath.Join(os.Getenv(\"STORAGE_PATH\"), id.NodeName)\n    }\n\n    return nil\n}\n```\n\n**Key Points**:\n- `ctx`: Context with tracing span and cancellation\n- `output`: Callback function to send data to other ports\n- `port`: Name of the port that received the message\n- `message`: The actual data (type assert to your config struct)\n- Return value is currently unused\n\n#### Configuration Schemas\n\nThe SDK automatically generates JSON Schemas from your Go structs. Use struct tags to control the UI:\n\n```go\ntype Config struct {\n    // Basic field\n    Name string `json:\"name\"`\n\n    // Configurable in UI (can reference other node outputs)\n    UserID string `json:\"userId\" configurable:\"true\"`\n\n    // Shared definition (other nodes can reference this)\n    Result string `json:\"result\" shared:\"true\"`\n\n    // Control UI layout\n    APIKey string `json:\"apiKey\" propertyOrder:\"1\" tab:\"auth\"`\n\n    // Nested object\n    Settings struct {\n        Timeout int `json:\"timeout\" configurable:\"true\"`\n    } `json:\"settings\"`\n\n    // Array\n    Items []string `json:\"items\" configurable:\"true\"`\n}\n```\n\n**Struct Tags**:\n- `configurable:\"true\"`: Field can accept values from other nodes via `{{expression}}` syntax\n- `shared:\"true\"`: Field definition is available to other nodes for type-safe mapping\n- `propertyOrder:\"N\"`: Controls field order in UI\n- `tab:\"name\"`: Groups field under a tab in UI\n- `align:\"horizontal\"`: Layout hint for UI\n\n#### System Ports\n\nSpecial ports available to all components:\n\n##### `_reconcile` Port\nCalled periodically (every 5 minutes) and on node changes:\n\n```go\ncase \"_reconcile\":\n    // Clean up resources\n    // Sync state\n    // Check for drift\n```\n\n##### `_client` Port\nProvides Kubernetes client for resource operations:\n\n```go\ncase \"_client\":\n    client := message.(resource.Manager)\n\n    // Create a signal\n    client.CreateSignal(ctx, resource.CreateSignalRequest{\n        Node: \"target-node\",\n        Port: \"input\",\n        Data: map[string]any{\"key\": \"value\"},\n    })\n    \n    // Get node information\n    node, err := client.GetNode(ctx, \"node-name\")\n```\n\n##### `_identity` Port\nReceives node identity so components can namespace local resources (e.g., paths on a shared PVC):\n\n```go\ncase v1alpha1.IdentityPort:\n    id, ok := message.(v1alpha1.NodeIdentity)\n    if !ok {\n        return fmt.Errorf(\"invalid identity\")\n    }\n    // Use id.NodeName to create a unique storage path\n    c.storagePath = filepath.Join(os.Getenv(\"STORAGE_PATH\"), id.NodeName)\n```\n\n`NodeIdentity` fields: `NodeName`, `Namespace`, `FlowName`, `ProjectName`.\n\n##### `_settings` Port\nReceives initial configuration (no \"from\" connection required):\n\n```go\ncase \"_settings\":\n    settings := message.(*MyConfig)\n    // Store settings for later use\n```\n\n#### Port Delivery Ordering\n\n**There is no guaranteed delivery order between system ports.** The `_settings`, `_reconcile`, `_control`, and `_identity` ports may fire in any sequence after a pod restart or during normal operation. Module creators must handle all possible orderings.\n\nCommon pitfall: a component restores state from metadata via `_reconcile`, then a `_settings` delivery arrives with stale CRD values and overwrites it. The SDK does not enforce any ordering — **it is the module creator's responsibility to handle this**.\n\nExample: if your component stores user-provided context via `_control` and persists it to metadata, you must protect it from being overwritten by `_settings`:\n\n```go\ntype Component struct {\n    settings         Settings\n    contextFromControl bool // tracks whether context was set by control/metadata\n}\n\ncase v1alpha1.SettingsPort:\n    in := msg.(Settings)\n    if c.contextFromControl {\n        in.Context = c.settings.Context // preserve control-set context\n    }\n    c.settings = in\n\ncase v1alpha1.ReconcilePort:\n    // restore from metadata\n    if cfg, ok := restoreFromMetadata(node); ok {\n        c.settings = cfg\n        c.contextFromControl = true\n    }\n\ncase v1alpha1.ControlPort:\n    ctrl := msg.(Control)\n    c.settings.Context = ctrl.Context\n    c.contextFromControl = true\n```\n\n#### Error Handling\n\n##### Transient Errors\nReturn regular errors for automatic retry with exponential backoff:\n\n```go\nfunc (c *MyComponent) Handle(ctx context.Context, output module.Handler, port string, msg any) any {\n    data, err := fetchFromAPI()\n    if err != nil {\n        // Will retry automatically\n        return err\n    }\n    // ...\n}\n```\n\n##### Permanent Errors\nUse `PermanentError` to stop retries:\n\n```go\nimport \"github.com/tiny-systems/module/pkg/errors\"\n\nfunc (c *MyComponent) Handle(ctx context.Context, output module.Handler, port string, msg any) any {\n    if !isValid(msg) {\n        // Won't retry - send to error port instead\n        output(ctx, \"error\", errors.PermanentError{\n            Err: fmt.Errorf(\"invalid input\"),\n        })\n        return nil\n    }\n    // ...\n}\n```\n\n#### Using the Resource Manager\n\nAccess Kubernetes resources from your component:\n\n```go\nimport \"github.com/tiny-systems/module/pkg/resource\"\n\nfunc (c *MyComponent) Handle(ctx context.Context, output module.Handler, port string, msg any) any {\n    if port == \"_client\" {\n        c.client = msg.(resource.Manager)\n        return nil\n    }\n\n    if port == \"create-flow\" {\n        // Create a new flow\n        flow, err := c.client.CreateFlow(ctx, resource.CreateFlowRequest{\n            Name:    \"dynamic-flow\",\n            Project: \"my-project\",\n        })\n\n        // Create nodes in the flow\n        node, err := c.client.CreateNode(ctx, resource.CreateNodeRequest{\n            Name:      \"node-1\",\n            Flow:      flow.Name,\n            Module:    \"http-module\",\n            Component: \"request\",\n            Settings: map[string]any{\n                \"url\": \"https://api.example.com\",\n            },\n        })\n\n        // Trigger the node\n        c.client.CreateSignal(ctx, resource.CreateSignalRequest{\n            Node: node.Name,\n            Port: \"trigger\",\n            Data: map[string]any{},\n        })\n    }\n\n    return nil\n}\n```\n\n#### Observability\n\nOpenTelemetry is built-in. The context includes a span:\n\n```go\nimport \"go.opentelemetry.io/otel\"\n\nfunc (c *MyComponent) Handle(ctx context.Context, output module.Handler, port string, msg any) any {\n    // Get tracer\n    tracer := otel.Tracer(\"my-module\")\n\n    // Create child span\n    ctx, span := tracer.Start(ctx, \"processing\")\n    defer span.End()\n\n    // Add attributes\n    span.SetAttributes(\n        attribute.String(\"input.size\", \"large\"),\n    )\n\n    // Do work...\n    result := doWork(ctx)\n\n    output(ctx, \"output\", result)\n    return nil\n}\n```\n\n#### Testing Components\n\n```go\npackage components_test\n\nimport (\n    \"context\"\n    \"testing\"\n    \"github.com/myorg/my-module/components\"\n)\n\nfunc TestHello(t *testing.T) {\n    component := \u0026components.Hello{}\n\n    var outputData *components.HelloOutput\n    outputHandler := func(ctx context.Context, port string, data any) any {\n        if port == \"output\" {\n            outputData = data.(*components.HelloOutput)\n        }\n        return nil\n    }\n\n    // Send message to input port\n    component.Handle(context.Background(), outputHandler, \"input\", \u0026components.HelloInput{\n        Name: \"World\",\n    })\n\n    // Verify output\n    if outputData.Greeting != \"Hello, World!\" {\n        t.Errorf(\"Expected 'Hello, World!', got '%s'\", outputData.Greeting)\n    }\n}\n```\n\n#### Best Practices\n\n1. **Keep Components Focused**: Each component should do one thing well\n2. **Use System Ports**: Implement `_reconcile` for cleanup\n3. **Handle Context Cancellation**: Respect `ctx.Done()` for graceful shutdown\n4. **Leverage Schemas**: Use struct tags to create great UI experiences\n5. **Share Definitions**: Mark output fields as `shared:true` for type-safe flows\n6. **Use Permanent Errors**: Don't retry validation errors or user mistakes\n7. **Add Observability**: Create spans for long operations\n8. **Document with Tags**: Use meaningful tags in `GetInfo()` for discoverability\n9. **Always Return Handler Results**: See critical section below\n\n#### CRITICAL: Handler Response Propagation\n\n**ALWAYS return the result of `handler()` calls. Never ignore the return value.**\n\nThe Tiny Systems SDK uses blocking I/O for request-response patterns. When a component like HTTP Server sends a request, it **blocks** waiting for a response to flow back through the same handler chain. If any component in the chain ignores the handler return value, the response is lost and the original caller times out.\n\n**BAD - Breaks blocking I/O:**\n```go\nfunc (c *Component) handleError(ctx context.Context, handler module.Handler, req Request, errMsg string) any {\n    if c.settings.EnableErrorPort {\n        _ = handler(ctx, \"error\", Error{...})  // WRONG: ignores return value!\n        return nil  // Response is lost, HTTP Server times out\n    }\n    return errors.New(errMsg)\n}\n```\n\n**GOOD - Propagates response correctly:**\n```go\nfunc (c *Component) handleError(ctx context.Context, handler module.Handler, req Request, errMsg string) any {\n    if c.settings.EnableErrorPort {\n        return handler(ctx, \"error\", Error{...})  // CORRECT: returns handler result\n    }\n    return errors.New(errMsg)\n}\n```\n\n**Why this matters:**\n\nWhen HTTP Server → Slack Command → Router → HTTP Server:Response:\n\n1. HTTP Server blocks on `handler(ctx, \"request\", req)`\n2. Message flows to Slack Command via gRPC\n3. Slack Command processes and calls `handler(ctx, \"error\", error)`\n4. Edge transforms Error → Response and sends to HTTP Server's response port\n5. HTTP Server's `handleResponse()` returns the Response\n6. **Response must flow back through the entire chain** to unblock HTTP Server\n\nIf Slack Command does `_ = handler(...)` and returns `nil`, the Response is lost.\n\n**Rules:**\n- Always `return handler(ctx, port, data)` for output ports\n- Exception: `_reconcile` and `_identity` port calls can ignore returns (internal system ports)\n- Exception: Fire-and-forget async operations where no response is expected\n\n#### Component Naming Convention\n\nAll component names must follow a consistent naming pattern using `snake_case`:\n\n**Pattern:** `[technology_][resource]_[action]` or `[resource]_[action]`\n\n**Rules:**\n1. Always use `snake_case` (lowercase with underscores)\n2. Resource/noun comes before action/verb\n3. Include technology prefix when multiple implementations exist\n4. Keep names concise (2-3 words max)\n\n**Examples by category:**\n\n| Category | Name | Description |\n|----------|------|-------------|\n| **HTTP** | `http_server` | HTTP server |\n| | `http_request` | Make HTTP request |\n| | `http_auth_parse` | Parse auth header |\n| **Encoding** | `json_encode` | Encode to JSON |\n| | `json_decode` | Decode from JSON |\n| | `go_template` | Render Go template |\n| **Email** | `smtp_send` | Send email via SMTP |\n| | `sendgrid_send` | Send email via SendGrid API |\n| **Messaging** | `slack_send` | Send Slack message |\n| | `slack_command` | Receive Slack command |\n| **Kubernetes** | `pod_status_get` | Get pod status |\n| | `pod_logs_get` | Get pod logs |\n| | `deployment_restart` | Restart deployment |\n| | `resource_watch` | Watch K8s resources |\n| **Utilities** | `transform` | Transform/passthrough data |\n| | `delay` | Delay execution |\n| | `router` | Route messages |\n\n**When to include technology prefix:**\n- Multiple implementations of same concept (e.g., `smtp_send` vs `sendgrid_send`)\n- Technology-specific behavior (e.g., `go_template` vs `handlebars_template`)\n- Clarity about protocol/API used (e.g., `grpc_call` vs `http_request`)\n\n#### Code Style\n\nFollow idiomatic Go patterns:\n\n1. **Early returns** - Avoid nested ifs, return early on errors\n2. **Flat structure** - Extract logic into small, focused functions\n3. **Error handling** - Use `if err != nil { return }` pattern\n4. **No deep nesting** - Maximum 2-3 levels of indentation\n\n```go\n// Good - early return\nfunc (c *Component) Handle(ctx context.Context, handler module.Handler, port string, msg any) error {\n    if port != \"request\" {\n        return fmt.Errorf(\"unknown port: %s\", port)\n    }\n\n    req, ok := msg.(Request)\n    if !ok {\n        return errors.New(\"invalid request\")\n    }\n\n    result, err := c.process(ctx, req)\n    if err != nil {\n        return c.handleError(ctx, handler, req, err)\n    }\n\n    return handler(ctx, \"output\", result)\n}\n\n// Bad - nested ifs\nfunc (c *Component) Handle(ctx context.Context, handler module.Handler, port string, msg any) error {\n    if port == \"request\" {\n        if req, ok := msg.(Request); ok {\n            if result, err := c.process(ctx, req); err == nil {\n                return handler(ctx, \"output\", result)\n            } else {\n                return c.handleError(ctx, handler, req, err)\n            }\n        }\n    }\n    return fmt.Errorf(\"unknown port: %s\", port)\n}\n```\n\n#### Component Design Principles\n\n**1. Minimize Settings**\n\nSettings should ONLY contain:\n- **Port flags** - `EnableErrorPort`, `EnableStatusPort` (affect port visibility)\n- **Precompiled resources** - Go templates, JS code (compiled once on change)\n- **Defaults** - Rarely-changing default values\n\nSettings should NOT contain:\n- Credentials (API keys, tokens, signing secrets)\n- Runtime URLs/endpoints\n- Per-request parameters\n- Anything that varies between executions\n\n```go\n// Good - minimal settings\ntype Settings struct {\n    EnableErrorPort bool  `json:\"enableErrorPort\" title:\"Enable Error Port\"`\n    DefaultLines    int64 `json:\"defaultLines\" title:\"Default Lines\"`\n}\n\n// Bad - credentials and runtime config in settings\ntype Settings struct {\n    APIKey      string `json:\"apiKey\"`       // Should be in request\n    Endpoint    string `json:\"endpoint\"`     // Should be in request\n    Namespace   string `json:\"namespace\"`    // Should be in request\n}\n```\n\n**2. Credentials via Input Ports**\n\nAll credentials and runtime configuration come through input ports:\n\n```go\ntype Request struct {\n    Context   any    `json:\"context,omitempty\" configurable:\"true\"`\n\n    // Credentials - from upstream (e.g., secret manager)\n    APIKey    string `json:\"apiKey\" required:\"true\" configurable:\"true\"`\n\n    // Runtime config - varies per execution\n    Endpoint  string `json:\"endpoint\" required:\"true\" configurable:\"true\"`\n    Namespace string `json:\"namespace\" required:\"true\" configurable:\"true\"`\n}\n```\n\n**Why:** Settings are spread across flows, not programmatically configurable, and storing credentials in settings is a security anti-pattern.\n\n**3. Context Passthrough**\n\nEvery component MUST pass context through for correlation:\n\n```go\ntype Request struct {\n    Context any `json:\"context,omitempty\" configurable:\"true\" title:\"Context\" description:\"Arbitrary data passed through to output\"`\n    // ... other fields\n}\n\ntype Response struct {\n    Context any `json:\"context,omitempty\" title:\"Context\"`\n    // ... other fields\n}\n\nfunc (c *Component) Handle(ctx context.Context, handler module.Handler, port string, msg any) error {\n    req := msg.(Request)\n\n    result := c.process(req)\n\n    // Always pass context through\n    handler(ctx, \"output\", Response{\n        Context: req.Context,  // Pass through!\n        Data:    result,\n    })\n    return nil\n}\n```\n\n**4. Flow-Driven Configuration**\n\nEverything should be configurable via edges/signals, not the settings panel:\n- Credentials mapped from upstream components (secret managers, vaults)\n- Runtime parameters from user input\n- Makes flows self-contained, portable, and version-controllable\n\n#### Example Modules\n\nCheck out these example modules for reference:\n- **common-module**: Basic utilities (delay, switch, merge, signal)\n- **http-module**: HTTP client/server components\n- **grpc-module**: gRPC service components\n\n#### CLI Reference\n\nThe SDK includes a CLI for running and building modules:\n\n```bash\n# Run module locally\n./my-module run --name=my-module --version=1.0.0 --namespace=tinysystems\n\n# Build (if custom build logic is needed)\n./my-module build\n\n# Get help\n./my-module --help\n```\n\n## License\n\nCopyright 2026 Tiny Systems Limited. All rights reserved.\n\nThis project is licensed under the [Business Source License 1.1](LICENSE).\n\n**Key terms:**\n- Free for production and non-production use\n- Not permitted: offering this software as a managed service, cloud service, or SaaS\n- Converts to Apache License 2.0 on January 11, 2031\n\nFor commercial licensing inquiries, contact Tiny Systems Limited.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftiny-systems%2Fmodule","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftiny-systems%2Fmodule","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftiny-systems%2Fmodule/lists"}