https://github.com/landerox/cloud-landerox-data

Reference architecture baseline for GCP data platforms (Apache Beam, BigQuery, Cloud Functions, Pub/Sub). Hybrid warehouse/lakehouse with batch + streaming, Medallion layering. Consumed by private runtime repos.
https://github.com/landerox/cloud-landerox-data

apache-beam batch-processing bigquery cloud-functions cloud-storage data-engineering data-platform dataform gcp google-cloud-dataflow iceberg lakehouse medallion-architecture opentelemetry pubsub python reference-architecture slsa streaming supply-chain-security

Last synced: 5 days ago
JSON representation

Reference architecture baseline for GCP data platforms (Apache Beam, BigQuery, Cloud Functions, Pub/Sub). Hybrid warehouse/lakehouse with batch + streaming, Medallion layering. Consumed by private runtime repos.

• Host: GitHub
• URL: https://github.com/landerox/cloud-landerox-data
• Owner: landerox
• License: mit
• Created: 2026-01-09T20:20:15.000Z (5 months ago)
• Default Branch: main
• Last Pushed: 2026-05-20T03:12:23.000Z (6 days ago)
• Last Synced: 2026-05-20T06:05:17.206Z (5 days ago)
• Topics: apache-beam, batch-processing, bigquery, cloud-functions, cloud-storage, data-engineering, data-platform, dataform, gcp, google-cloud-dataflow, iceberg, lakehouse, medallion-architecture, opentelemetry, pubsub, python, reference-architecture, slsa, streaming, supply-chain-security
• Language: Python
• Homepage: https://landerox.com/projects/cloud-data
• Size: 245 KB
• Stars: 1
• Watchers: 0
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • Contributing: .github/CONTRIBUTING.md
  • License: LICENSE
  • Code of conduct: .github/CODE_OF_CONDUCT.md
  • Codeowners: .github/CODEOWNERS
  • Security: .github/SECURITY.md
  • Agents: AGENTS.md

Awesome Lists containing this project

README

          
# cloud-landerox-data

[![CI](https://github.com/landerox/cloud-landerox-data/actions/workflows/lint.yml/badge.svg)](https://github.com/landerox/cloud-landerox-data/actions/workflows/lint.yml)

[![CodeQL](https://github.com/landerox/cloud-landerox-data/actions/workflows/codeql.yml/badge.svg)](https://github.com/landerox/cloud-landerox-data/actions/workflows/codeql.yml)

[![codecov](https://codecov.io/gh/landerox/cloud-landerox-data/branch/main/graph/badge.svg)](https://codecov.io/gh/landerox/cloud-landerox-data)

[![OpenSSF Scorecard](https://api.scorecard.dev/projects/github.com/landerox/cloud-landerox-data/badge)](https://scorecard.dev/viewer/?uri=github.com/landerox/cloud-landerox-data)

[![OpenSSF Best Practices](https://www.bestpractices.dev/projects/12905/badge)](https://www.bestpractices.dev/projects/12905)

[![OpenSSF Baseline](https://www.bestpractices.dev/projects/12905/baseline)](https://www.bestpractices.dev/projects/12905)

[![SLSA Level 3](https://slsa.dev/images/gh-badge-level3.svg)](https://slsa.dev)

[![Python 3.13](https://img.shields.io/badge/python-3.13-3776AB?logo=python&logoColor=white)](https://www.python.org/downloads/release/python-3130/)

[![Checked with pyright](https://img.shields.io/badge/pyright-checked-0E7FC0?logo=python&logoColor=white)](https://github.com/microsoft/pyright)

[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)

[![Conventional Commits](https://img.shields.io/badge/Conventional%20Commits-1.0.0-FE5196?logo=conventionalcommits&logoColor=white)](https://www.conventionalcommits.org)

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

[![GCP](https://img.shields.io/badge/GCP-Ready-4285F4?logo=google-cloud&logoColor=white)](https://cloud.google.com/)

> **Reference architecture baseline for GCP data platforms** — hybrid

> warehouse/lakehouse, batch + streaming, Medallion layering.

> Consumed by private runtime repos.

Welcome to the `cloud-landerox-data` repository. This project serves as a comprehensive architecture baseline for Google Cloud Platform (GCP) data platforms. It provides structural guidance, engineering standards, and folder placeholders designed to be consumed by private runtime repositories.

Please note that this repository is **not** a production runtime environment, a Terraform infrastructure repository, or a Data Mesh framework. Instead, it is a pragmatic, hybrid baseline that intelligently selects patterns based on data source, Service Level Agreements (SLAs), and cost profiles. For a deep dive into our decision-making model, please review our [Architecture Stance](docs/architecture.md).

## Status

The project is currently in a **pre-1.0** state with active quality and supply-chain gates on every PR and push: ruff, pyright (strict), pytest with coverage (≥90% on `shared/**`), `hypothesis` property tests, nightly `mutmut`; CodeQL, Semgrep, Trivy, pip-audit, license allowlist, gitleaks, zizmor, validate-pyproject, lychee link check, and DCO sign-off enforced locally and in CI. OpenSSF Scorecard runs weekly. Every release attaches a CycloneDX SBOM and a SLSA L3 build provenance attestation. Dependabot regular and security updates are both enabled. By design, no runtime code is deployed from this repository.

To see how this baseline is applied in practice, check out our minimal public runtime example at [landerox/cloud-landerox-runtime-example](https://github.com/landerox/cloud-landerox-runtime-example).

## Getting started

### Prerequisites

- Python 3.13

- [`mise`](https://mise.jdx.dev/getting-started.html) — manages `just`,

  `lychee`, and other non-Python tools via `mise.toml`

- [`uv`](https://docs.astral.sh/uv/)

### Setup

```bash

mise install

just sync

just pre-commit-install

```

### Quality checks

```bash

just lint

just type

just test

```

## Learn more

- [Documentation index](docs/README.md) — full catalogue of architecture, guides, diagrams, and blueprints

- [Architecture stance](docs/architecture.md)

- [Architecture decisions (ADRs)](docs/adr/README.md)

- [CI/CD guide](docs/cicd.md)

- [Governance](GOVERNANCE.md)

- [Roadmap](ROADMAP.md)

- [Contributing](.github/CONTRIBUTING.md)

- [Security policy](.github/SECURITY.md)

- [Changelog](CHANGELOG.md)

## License

This project is licensed under the terms found in the [LICENSE](LICENSE) file.