{"id":50472353,"url":"https://github.com/semcod/protos","last_synced_at":"2026-06-01T11:03:12.652Z","repository":{"id":353525109,"uuid":"1219791412","full_name":"semcod/protos","owner":"semcod","description":null,"archived":false,"fork":false,"pushed_at":"2026-05-13T15:10:13.000Z","size":4904,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-13T17:17:34.788Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/semcod.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-24T08:13:40.000Z","updated_at":"2026-05-13T15:37:21.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/semcod/protos","commit_stats":null,"previous_names":["semcod/protos"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/semcod/protos","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semcod%2Fprotos","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semcod%2Fprotos/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semcod%2Fprotos/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semcod%2Fprotos/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/semcod","download_url":"https://codeload.github.com/semcod/protos/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semcod%2Fprotos/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33771630,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-01T02:00:06.963Z","response_time":115,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-01T11:03:12.366Z","updated_at":"2026-06-01T11:03:12.642Z","avatar_url":"https://github.com/semcod.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# protogate\n\nMigration tool and delegation platform for extracting bounded slices from legacy systems with minimal coupling. Built on SUMD + DOQL + testql + taskfile ecosystem.\n\nPackage-level operational guide: see protogate/README.md.\n\n## Architecture\n\n**c2004 owns (c2004-first):**\n\n- Contracts (Protobuf)\n- Generators and schema registry\n- Commands \u0026 Queries (CQRS) handlers\n- Migration discovery and planning artifacts\n- Shell, navigation, auth/session bridge, iframe routing\n\n**protogate owns:**\n\n- Delegation/execution tooling layer\n- Runtime bridge for invoking migration tooling from c2004\n- Gateway runtime and health endpoints\n\nprotogate is not the source-of-truth for migration contracts, discovery logic, or planning artifacts.\n\nEach delegated module follows a vertical-slice template:\n\n- `contracts/{slice}/v1/` - Protobuf contracts\n- `gateway/{slice}_handler.py` - Command/query handlers\n- Event store + read model adapters\n- Frontend assets in `gateway/static/`\n- Smoke tests \u0026 health endpoints\n\n## Quick Start\n\n### Using CLI (Recommended)\n\n```bash\n# Install protogate CLI\npip install -e .\n\n# Generate all artifacts from contracts\nprotogate generate all\n\n# Run specific generator\nprotogate generate python\nprotogate generate zod\n\n# Schema registry operations\nprotogate registry check contracts/user/v1/user.proto\nprotogate registry list\n\n# Run gateway\nprotogate gateway\n\n# Run full CI pipeline\nprotogate ci\n```\n\n### Using Makefile (Legacy)\n\n```bash\n# Install dependencies\npip install -r requirements.txt\n\n# Run gateway (development mode)\nmake gateway\n\n# Run full CI pipeline\nmake ci\n```\n\n## API Overview\n\n### Core Endpoints\n\n| Method | Endpoint | Description |\n| ------ | -------- | ----------- |\n| GET | `/health` | Platform health + module aggregation |\n| GET | `/health/modules` | All delegated slices status |\n| GET | `/health/modules/{slice}` | Specific slice health |\n| GET | `/delegation/slices` | List all delegated slices |\n| GET | `/delegation/slices/{slice}` | Slice details \u0026 metadata |\n\n### User Module (Live)\n\n| Method | Endpoint | Description |\n| ------ | -------- | ----------- |\n| POST | `/commands/user/create` | Create user |\n| POST | `/commands/user/dual-create` | Dual-write with idempotency |\n| POST | `/commands/user/{id}/change-email` | Change email |\n| POST | `/commands/user/{id}/deactivate` | Deactivate user |\n| GET | `/queries/user/{id}` | Get user state |\n| GET | `/events` | Event stream |\n\n### Search Module (Phase-1)\n\n| Method | Endpoint | Description |\n| ------ | -------- | ----------- |\n| POST | `/commands/search/index` | Index entry |\n| GET | `/queries/search?q={query}` | Full-text search |\n\n## Code Generation\n\nThe project provides multiple code generators from Protobuf contracts:\n\n### Makefile Targets\n\n| Target | Description |\n| ------ | ----------- |\n| `make proto` | Generate gRPC stubs via buf (requires buf CLI) |\n| `make zod` | Generate TypeScript Zod schemas |\n| `make python` | Generate Pydantic Python models |\n| `make json` | Generate JSON Schema (draft-07) |\n| `make sql` | Generate SQL DDL |\n| `make proto-all` | Run all generators (proto + zod + python + json + sql) |\n| `make proto-changed` | Detect changed proto files against main branch |\n| `make generate-incremental` | Incremental regeneration (only changed proto files) |\n| `make clean` | Remove all generated artifacts |\n\n### Schema Registry\n\nManage schema versions and compatibility:\n\n| Target | Description |\n| ------ | ----------- |\n| `make registry-register` | Register proto file in schema registry |\n| `make registry-check` | Check compatibility without registering |\n| `make registry-list` | List all schemas in registry |\n\n### Legacy Bridge\n\nLegacy schema migration and synchronization:\n\n| Target | Description |\n| ------ | ----------- |\n| `make legacy-register` | Register legacy JSON schema + proto mapping |\n| `make diff-legacy` | Diff legacy vs proto schemas |\n| `make legacy-report` | Generate detailed migration report |\n| `make legacy-list` | List all legacy schemas |\n| `make sync-check` | Full sync check (fails if readiness \u003c 1.0) |\n| `make bootstrap-legacy` | Bootstrap EventStore from legacy DB |\n\n### CI Pipeline\n\n| Target | Description |\n| ------ | ----------- |\n| `make ci` | Full CI: lint → generate → test → registry check |\n\n## Contract Enum Cross-Check\n\n`protogate codegen registry` cross-checks CQRS contract JSON files\n(`*.command.json`, `*.query.json`, `*.event.json`) against Pydantic\n`Literal[...]` annotations found in `layers.python`. It catches the\ndrift class that broke ADR-012 Wave 2 in c2004, where a server could\nreturn an enum value the contract did not advertise, crashing the\nclient decoder.\n\n### CLI\n\n```bash\n# 1. Report-only gate (CI default)\nprotogate codegen registry contracts/ --check --cross-check-pydantic\n\n# 2. Auto-fix warnings (safe; touches JSON only)\nprotogate codegen registry contracts/ --cross-check-pydantic --fix-safe\n\n# 3. Auto-fix warnings + expand output enums (opt-in; may bless server bugs)\nprotogate codegen registry contracts/ --cross-check-pydantic --fix-safe --auto-expand-output\n```\n\n### Directional rules\n\n| Direction | Rule | Verdict |\n| --- | --- | --- |\n| `output`/`payload` | `pydantic ⊆ contract` | compatible |\n| `output`/`payload` | `pydantic ⊋ contract` | **error** — client may crash on undeclared value |\n| `output`/`payload` | `contract ⊋ pydantic` | **warning** — dead code paths on client |\n| `input`            | `contract ⊆ pydantic` | compatible |\n| `input`            | `contract ⊋ pydantic` | **error** — server rejects valid-per-contract input (HTTP 422) |\n| `input`            | `pydantic ⊋ contract` | compatible (intentional API restriction) |\n\nPydantic source is **never** modified; all fixes apply to contract JSON\nonly. Input-direction errors are never auto-fixed (require human\ndecision between narrowing the contract or loosening Pydantic).\n\nSee [docs/contract-cross-check.md](docs/contract-cross-check.md) for the\nfull reference, the c2004 Makefile integration, and the Wave 2\npost-mortem that motivated the feature.\n\n## Delegation Workflow\n\n1. **Generate candidate report** in c2004 (`detect_migration_candidates.py`)\n2. **Generate delegation plan** in protogate:\n\n   ```bash\n   python scripts/legacy_bridge/generate_delegation_plan.py \\\n     --input /path/to/c2004/module-candidates.json \\\n     --clusters /path/to/c2004/cqrs-pattern-clusters.json \\\n     --output-dir docs\n   ```\n\n3. **(Recommended) Run full discovery pipeline** in protogate:\n\n   ```bash\n   python scripts/legacy_bridge/run_arch_migration_discovery.py \\\n     --repo-root /path/to/c2004 \\\n     --output-dir reports/migration-discovery \\\n     --delegation-limit 30\n   ```\n\n4. Pick top module from Phase-1\n5. Implement full vertical slice in protogate\n6. Switch c2004 route to iframe host\n7. Validate parity \u0026 archive legacy\n\n## Project Structure\n\n```text\nprotogate/\n├── contracts/              # Protobuf contracts per slice\n│   ├── user/v{1,2}/\n│   ├── search/v1/\n│   └── legacy_bridge/\n├── gateway/                # FastAPI gateway\n│   ├── main.py            # Entry point \u0026 routes\n│   ├── delegation.py      # Slice registry \u0026 health\n│   ├── user_handler.py    # User CQRS handlers\n│   ├── search_handler.py  # Search CQRS handlers\n│   └── static/            # Delegated UI assets\n├── adapters/              # Legacy ↔ Proto adapters\n├── scripts/               # Code generation \u0026 migration\n│   ├── generate_zod.py    # TypeScript Zod generator\n│   ├── generate_pydantic.py # Python Pydantic generator\n│   ├── generate_json_schema.py # JSON Schema generator\n│   ├── generate_sql.py    # SQL DDL generator\n│   ├── generate_incremental.py # Incremental regeneration\n│   ├── schema_registry.py # Proto schema registry\n│   ├── legacy_registry.py # Legacy schema registry\n│   ├── event_store.py     # CQRS event store\n│   ├── conflict_resolver.py # Event conflict resolution\n│   ├── dual_writer.py     # Dual-write pattern\n│   ├── idempotency_store.py # Idempotency tracking\n│   ├── vector_clock.py    # Vector clock for ordering\n│   └── legacy_bridge/     # Migration tooling\n│       ├── run_arch_migration_discovery.py # Full orchestrator\n│       ├── detect_migration_candidates.py # Module scoring\n│       ├── analyze_service_boundaries.py # Frontend/backend analysis\n│       ├── detect_cqrs_pattern_clusters.py # CQRS pattern detection\n│       ├── generate_migration_wave_plan.py # Wave planning\n│       ├── delegation_plan.py # Delegation plan logic\n│       ├── generate_delegation_plan.py # Plan generator\n│       ├── migrator.py # Legacy to EventStore migration\n│       ├── sync_check.py # Sync validation\n│       └── diff_engine.py # Schema diffing\n├── tests/                 # Test suite\n└── docs/                  # Generated plans\n    ├── delegation-plan.generated.json\n    └── delegation-plan.generated.md\n```\n\n## Key Components\n\n### Code Generators\n\n- **Zod Generator** (`scripts/generate_zod.py`): TypeScript runtime validation schemas\n- **Pydantic Generator** (`scripts/generate_pydantic.py`): Python data models\n- **JSON Schema Generator** (`scripts/generate_json_schema.py`): Draft-07 JSON schemas\n- **SQL Generator** (`scripts/generate_sql.py`): Database DDL\n- **Incremental Generator** (`scripts/generate_incremental.py`): Regenerate only changed proto files\n\n### Schema Registry\n\nSQLite-backed schema registry with compatibility enforcement (`scripts/schema_registry.py`):\n\n- Register schema versions with SHA256 hashing\n- Check backward/forward compatibility\n- List all registered schemas\n- Prevent breaking changes\n\n### Legacy Bridge\n\nComprehensive migration tooling for legacy systems:\n\n- **Migration Discovery Orchestrator**: Full pipeline profiling, candidate detection, service boundary analysis, CQRS pattern clustering, and delegation planning\n- **Migration Candidate Detection**: Score modules by extraction suitability, identify service boundaries\n- **Service Boundary Analysis**: Frontend module detection, backend route grouping, iframe suitability assessment\n- **CQRS Pattern Clustering**: Classify modules by command/event patterns (data-grid, reports, manager, config)\n- **Migration Wave Planning**: Generate phased extraction plans with effort estimation\n- **Legacy Schema Registry**: Track legacy JSON schemas and proto mappings\n- **Diff Engine**: Compare legacy vs proto schemas for compatibility\n- **Migrator**: Bootstrap EventStore from legacy databases\n- **Sync Check**: Validate legacy-proto synchronization readiness\n\n### CQRS Infrastructure\n\n- **Event Store** (`scripts/event_store.py`): Append-only event store with SQLite, optimistic concurrency, snapshots, stream merging\n- **Conflict Resolver** (`scripts/conflict_resolver.py`): Last-Write-Wins and merge strategies for concurrent events\n- **Vector Clock** (`scripts/vector_clock.py`): Causal ordering and conflict detection\n- **Dual Writer** (`scripts/dual_writer.py`): Dual-write pattern for legacy migration\n- **Idempotency Store** (`scripts/idempotency_store.py`): Prevent duplicate command processing\n\n### DelegatedSlice Registry\n\nRuntime model for slice metadata in `gateway/delegation.py`:\n\n```python\nDelegatedSlice(\n    name=\"search\",\n    phase=\"phase-1\",           # phase-1 | phase-2 | live\n    backend=\"delegated\",\n    frontend=\"static\",         # none | static | planned\n    contract_paths=(\"contracts/search/v1/search.proto\",),\n    command_routes=(\"/commands/search/index\",),\n    query_routes=(\"/queries/search\",),\n    smoke_checks=(\"/health\", \"/queries/search?q=test\"),\n)\n```\n\n### Health Checks\n\nPer-slice health validates:\n\n- Contract files exist\n- Read model assets present\n- Frontend assets (if required)\n\nReturns `ok` or `degraded` with missing requirements listed.\n\n## Deployment\n\n### Docker Compose\n\n```bash\n# Build and run all services\ndocker-compose up\n\n# Services:\n# - generator: Proto code generation\n# - gateway: FastAPI gateway (port 8080)\n```\n\n### Gateway Docker\n\n```bash\n# Build gateway image\ndocker build -f gateway/Dockerfile -t semcod-gateway .\n\n# Run gateway container\ndocker run --rm -p 8080:8080 semcod-gateway\n```\n\nOr use Makefile:\n\n```bash\nmake gateway-docker\n```\n\n## Environment Variables\n\n| Variable | Default | Description |\n|----------|---------|-------------|\n| `OPENROUTER_API_KEY` | `*(not set)*` | OpenRouter API key (https://openrouter.ai/keys) |\n| `LLM_MODEL` | `openrouter/qwen/qwen3-coder-next` | LLM model for AI-assisted features |\n| `PFIX_AUTO_APPLY` | `true` | Apply fixes without asking |\n| `PFIX_AUTO_INSTALL_DEPS` | `true` | Auto pip/uv install dependencies |\n| `PFIX_AUTO_RESTART` | `false` | Restart after fix |\n| `PFIX_MAX_RETRIES` | `3` | Max retry attempts |\n| `PFIX_ENABLED` | `true` | Enable auto-fix features |\n| `PFIX_GIT_COMMIT` | `false` | Auto-commit fixes |\n| `PFIX_GIT_PREFIX` | `pfix:` | Commit message prefix |\n\n## Testing\n\n### TestQL Scenarios\n\nAuto-generated API smoke tests in `testql-scenarios/generated-api-smoke.testql.toon.yaml`:\n\n- Health checks\n- Delegation slice endpoints\n- Command/query endpoints\n- Event streaming\n\n### Pytest\n\n```bash\n# Run all tests\npytest tests/ -v\n\n# Run specific test\npytest tests/test_event_store.py -v\n```\n\n## Release Management\n\n- **Versioning**: Semantic versioning (semver)\n- **Commits**: Conventional commits with scope=`protogate`\n- **Changelog**: Keep-a-changelog format\n- **Build strategies**: Python, Node.js, Rust\n- **Version files**: `VERSION`, generated package versions\n\n## AI Cost Tracking\n\n![PyPI](https://img.shields.io/badge/pypi-costs-blue) ![Version](https://img.shields.io/badge/version-0.1.24-blue) ![Python](https://img.shields.io/badge/python-3.9+-blue) ![License](https://img.shields.io/badge/license-Apache--2.0-green)\n![AI Cost](https://img.shields.io/badge/AI%20Cost-$5.40-orange) ![Human Time](https://img.shields.io/badge/Human%20Time-16.2h-blue) ![Model](https://img.shields.io/badge/Model-openrouter%2Fqwen%2Fqwen3--coder--next-lightgrey)\n\n- 🤖 **LLM usage:** $5.4000 (36 commits)\n- 👤 **Human dev:** ~$1623 (16.2h @ $100/h, 30min dedup)\n\nGenerated on 2026-04-26 using [openrouter/qwen/qwen3-coder-next](https://openrouter.ai/qwen/qwen3-coder-next)\n\n---\n\n## License\n\nLicensed under Apache-2.0.\n## Status\n\n_Last updated by [taskill](https://github.com/oqlos/taskill) at 2026-04-25 13:43 UTC_\n\n| Metric | Value |\n|---|---|\n| HEAD | `3d1c764` |\n| Coverage | — |\n| Failing tests | — |\n| Commits in last cycle | 40 |\n\n\u003e Various tests, docs, and refactors plus new features for codegen/registry and protogate CLI. Additions include directional subset checks, a --cross-check-pydantic flag, migration-analysis CLI commands, and schema registry improvements (conflict resolution, vector clocks, v2 proto).\n\n\u003c!-- taskill:status:end --\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsemcod%2Fprotos","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsemcod%2Fprotos","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsemcod%2Fprotos/lists"}