{"id":51358666,"url":"https://github.com/k8shell-io/common","last_synced_at":"2026-07-02T21:34:00.571Z","repository":{"id":367837969,"uuid":"1043250239","full_name":"k8shell-io/common","owner":"k8shell-io","description":"Shared Go modules used across k8shell services","archived":false,"fork":false,"pushed_at":"2026-06-27T21:58:57.000Z","size":731,"stargazers_count":0,"open_issues_count":2,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-27T23:21:04.772Z","etag":null,"topics":["grpc-go"],"latest_commit_sha":null,"homepage":"https://docs.k8shell.io","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/k8shell-io.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-08-23T13:04:46.000Z","updated_at":"2026-06-26T10:21:11.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/k8shell-io/common","commit_stats":null,"previous_names":["k8shell-io/common"],"tags_count":396,"template":false,"template_full_name":null,"purl":"pkg:github/k8shell-io/common","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k8shell-io%2Fcommon","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k8shell-io%2Fcommon/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k8shell-io%2Fcommon/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k8shell-io%2Fcommon/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/k8shell-io","download_url":"https://codeload.github.com/k8shell-io/common/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/k8shell-io%2Fcommon/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35064249,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-02T02:00:06.368Z","response_time":173,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["grpc-go"],"created_at":"2026-07-02T21:33:59.156Z","updated_at":"2026-07-02T21:34:00.570Z","avatar_url":"https://github.com/k8shell-io.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# common\n\nShared Go library for the k8Shell platform. Defines all gRPC API contracts (proto files, generated stubs, typed clients), canonical domain models, and utility packages consumed by every k8Shell service.\n\n## Concepts\n\n### Proto contracts\n\nAll service interfaces are defined as Protocol Buffer files under `pkg/api/proto/`. Generated Go stubs live in `pkg/api/gen/go/` and are committed to the repository so consumers do not need a proto toolchain to build.\n\nThe library defines the following gRPC services:\n\n| Service | Proto file | Description |\n|---|---|---|\n| `IdentityService` | `identity/v1/identity.proto` | User lookup, authentication, onboarding, credential management, PAT lifecycle |\n| `IdentityProviderService` | `identity/v1/idp.proto` | Interface implemented by remote identity providers (GitHub, GitLab, etc.) |\n| `AuthzService` | `authz/v1/authz.proto` | OPA/Rego policy evaluation — single and batch authorization checks |\n| `ProvisionerService` | `provisioner/v1/provisioner.proto` | Workspace lifecycle (create, list, delete, stop) with streaming progress events |\n| `SessionService` | `session/v1/session.proto` | SSH session recording and metadata tracking |\n| `RecordingService` | `session/v1/session.proto` | Client-streaming ingestion of PTY, exec, port-forward, and SFTP channel recordings |\n| `SystemService` | `k8shelld/v1/k8shelld.proto` | In-workspace daemon: handshake and system metrics |\n| `SshService` | `k8shelld/v1/k8shelld.proto` | In-workspace daemon: shell, port-forward, exec, Unix socket operations |\n| `CommandService` | `k8shelld/v1/k8shelld.proto` | In-workspace daemon: bidirectional command messaging |\n| `AppService` | `k8shelld/v1/k8shelld.proto` | In-workspace daemon: app lifecycle (install, start, stop, logs) |\n\nTwo message-only protocols drive the web console:\n\n| Protocol | Proto file | Description |\n|---|---|---|\n| `WebShellMessage` | `console/webshell/v1/webshell.proto` | Terminal I/O over WebSocket |\n| `WebFilesMessage` | `console/webfiles/v1/webfiles.proto` | File explorer operations over WebSocket |\n| `CloudshellMessage` | `cloudshell/cloudshell.proto` | Unified web console envelope |\n\nShared message types used across services are defined in `common/v1/common.proto`.\n\n### Typed gRPC clients\n\n`pkg/api/client/` provides thin, preconfigured gRPC client wrappers for the four services most commonly dialed by other k8Shell components: `identity`, `k8shelld`, `provisioner`, and `session`. Each client handles TLS setup and connection lifetime, and exposes the generated service client directly.\n\n### Domain models\n\n`pkg/models` defines the canonical Go structs used throughout the platform. These are distinct from the generated Protobuf types and carry richer Go-native behavior (validation tags, helper methods).\n\n| Model | Key fields |\n|---|---|\n| `User` | Username, organization, POSIX UID/GID, roles, blueprint permissions, auth methods (password, SSH keys, OIDC providers), credential list |\n| `Workspace` | Status, resource allocation (CPU/memory), pod details (IP, hostname, port, namespace), repository context (owner, name, branch/tag/commit), blueprint and blueprint kind |\n| `SSHSession` | Session ID, user, client IP, timestamps, byte counts (input/output), channel types used |\n| `UserCredential` | External service credentials (registry, git, Kubernetes), OAuth tokens and API keys, expiration and activity timestamps |\n| `AccessToken` | Personal Access Token with scope list, expiration, and last-used timestamp |\n\n### Authz contracts\n\n`pkg/authz` defines typed request builders for every authorization policy point evaluated across the platform. Services construct a typed request with `With*` builder methods, call `ToProto` to produce the wire message, and send it to the `AuthzService`. On the server side, `FromProto` validates and restores the typed struct.\n\n| Policy | Who evaluates | Purpose |\n|---|---|---|\n| `user:onboard` | Identity service | Can deny onboarding or attach obligations (sudo, roles, blueprints) before a new user is persisted |\n| `user:auth` | Identity service | Evaluated on each authentication attempt |\n| `user:read` | Identity service | Controls visibility of individual user records |\n| `user:list` | Identity service | Controls whether a caller may enumerate users |\n| `token:create` | Identity service | Can restrict PAT scopes or cap maximum expiry |\n| `token:read` | Identity service | Controls access to PAT metadata |\n\nObligations returned by the policy engine (additional scopes, expiry caps, role assignments) are parsed by helpers in this package into strongly typed Go structs.\n\n### User string format\n\n`pkg/userstr` parses the SSH username string that carries workspace identity through the k8Shell SSH layer. A user string encodes the username plus optional workspace context in a compact, URL-safe format.\n\n| Form | Example | Meaning |\n|---|---|---|\n| Implicit | `alice` | Default workspace, implicit blueprint |\n| Explicit blueprint | `alice+myblueprint` | Named blueprint, no repository |\n| Named workspace | `alice@myworkspace` | Explicit workspace name |\n| Repo workspace | `alice/owner/repo[/ref]` | Repository-bound workspace (branch, tag, or commit) |\n\nStrings may be base64-encoded (prefixed `b64-`) to safely carry arbitrary repo paths through SSH client constraints. Blueprint kind (`implicit`, `explicit`, `custom`) is carried as a typed field on the parsed struct.\n\n### gRPC server utilities\n\n`pkg/gapi` provides a configurable gRPC server wrapper used by every k8Shell service:\n\n- **TLS** — certificate loading from disk with optional hot-reload on file change (configurable delay).\n- **OIDC/JWT authentication** — validates Bearer tokens against a configured issuer and audience. Bypassed for requests from trusted Kubernetes ServiceAccounts.\n- **Kubernetes ServiceAccount authorization** — allows calls from specific namespace/ServiceAccount pairs without a user JWT (for in-cluster service-to-service traffic).\n- **Request logging** — per-RPC log lines with method, duration, peer address, and caller identity.\n\n### Configuration processing\n\n`pkg/config` processes YAML configuration files before unmarshalling:\n\n- **Environment variable expansion** — `${VAR_NAME}` references are resolved at load time. Variables must be present unless `RequireEnvVars` is disabled.\n- **`!file` tag** — inlines the contents of an external file into a YAML string value, useful for mounting secrets from Kubernetes.\n- **`FileWatcher`** — wraps `fsnotify` to trigger a reload callback when the configuration file changes on disk.\n\n### NATS/JetStream client\n\n`pkg/nats` provides a managed NATS connection with:\n\n- Auto-reconnect with configurable backoff and maximum attempts (default: infinite).\n- JetStream context for persistent streams.\n- KV store helpers with TTL-based caching.\n- Distributed lock primitives backed by JetStream KV.\n- Generic JSON publish/subscribe helpers.\n\n### Structured logging\n\n`pkg/logger` wraps [zerolog](https://github.com/rs/zerolog) with a small initialization helper. Output is JSON by default; pass `--logtext` (or set the text flag in code) to enable human-readable console output. Every log line includes a component name and the process PID.\n\n### Database utilities\n\n`pkg/db` provides a thin connection helper over [pgx/v5](https://github.com/jackc/pgx) and exposes SQL migration support via [golang-migrate](https://github.com/golang-migrate/migrate). Services call `db.Connect` to get a `*pgxpool.Pool` from a standard `DATABASE_URL`-style DSN.\n\n## Repository layout\n\n```\npkg/\n  api/\n    proto/           # Protocol Buffer source files\n      authz/v1/\n      cloudshell/\n      common/v1/\n      console/\n        webfiles/v1/\n        webshell/v1/\n      identity/v1/\n      k8shelld/v1/\n      provisioner/v1/\n      session/v1/\n    gen/go/          # Generated Go stubs (committed, do not edit)\n    client/          # Typed gRPC client wrappers\n      identity/\n      k8shelld/\n      provisioner/\n      session/\n  authz/             # Typed authz policy request builders and obligation parsing\n  config/            # YAML processor (env-var expansion, !file tag, FileWatcher)\n  db/                # pgx connection helper and migration support\n  gapi/              # gRPC server wrapper (TLS, OIDC auth, request logging)\n  logger/            # zerolog initialization (JSON / console output)\n  models/            # Canonical domain models (User, Workspace, SSHSession, etc.)\n  nats/              # NATS/JetStream client (reconnect, KV cache, distributed lock)\n  userstr/           # SSH username / workspace identity string parser\n  utils/             # General-purpose helpers\n  validator/         # Input validation wrappers\n```\n\n## Prerequisites\n\n- Go 1.24+\n- [`buf`](https://buf.build/docs/installation) (recommended) **or** `protoc` with `protoc-gen-go` and `protoc-gen-go-grpc` plugins — only required to regenerate stubs from proto sources\n\n## Code generation\n\nGenerated stubs are committed and kept up to date in CI. You only need to run generation locally when modifying `.proto` files.\n\n**With buf (recommended):**\n```bash\ncd pkg/api/proto \u0026\u0026 buf generate\n```\n\n**With protoc (fallback):**\n```bash\nmake proto\n```\n\n**Lint proto files:**\n```bash\nmake proto-lint\n```\n\n**Check for breaking changes against `main`:**\n```bash\nmake proto-breaking\n```\n\n## Makefile targets\n\n| Target | Description |\n|---|---|\n| `make proto` | Generate Go stubs from all `.proto` files (uses `buf` if available, falls back to `protoc`) |\n| `make proto-lint` | Run `buf lint` over all proto files |\n| `make proto-breaking` | Check for breaking wire changes against the `main` branch |\n\n## Using as a dependency\n\n```bash\ngo get github.com/k8shell-io/common@latest\n```\n\nImport the package you need:\n\n```go\nimport (\n    identityv1  \"github.com/k8shell-io/common/pkg/api/gen/go/identity/v1\"\n    \"github.com/k8shell-io/common/pkg/api/client/identity\"\n    \"github.com/k8shell-io/common/pkg/models\"\n    \"github.com/k8shell-io/common/pkg/userstr\"\n)\n```\n\nServices that expose a gRPC endpoint embed `pkg/gapi.Server`; services that dial other k8Shell services use the typed clients in `pkg/api/client/`.\n\n## License\n\nAGPL-3.0-or-later. See [LICENSE](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fk8shell-io%2Fcommon","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fk8shell-io%2Fcommon","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fk8shell-io%2Fcommon/lists"}