{"id":44109915,"url":"https://github.com/vmariiechko/databricks-bundle-template","last_synced_at":"2026-06-01T07:32:39.909Z","repository":{"id":328118078,"uuid":"1086229425","full_name":"vmariiechko/databricks-bundle-template","owner":"vmariiechko","description":"A custom Declarative Automation Bundles template for production-ready, multi-environment Databricks data pipeline projects","archived":false,"fork":false,"pushed_at":"2026-05-09T14:27:48.000Z","size":875,"stargazers_count":5,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-09T16:35:33.418Z","etag":null,"topics":["ci-cd","dabs","data-engineering","databricks","declarative-automation-bundles","devops","template","unity-catalog"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vmariiechko.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":"ROADMAP.md","authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2025-10-30T06:01:28.000Z","updated_at":"2026-05-09T14:27:51.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/vmariiechko/databricks-bundle-template","commit_stats":null,"previous_names":["vmariiechko/databricks-bundles-realworld","vmariiechko/databricks-bundle-template"],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/vmariiechko/databricks-bundle-template","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vmariiechko%2Fdatabricks-bundle-template","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vmariiechko%2Fdatabricks-bundle-template/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vmariiechko%2Fdatabricks-bundle-template/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vmariiechko%2Fdatabricks-bundle-template/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vmariiechko","download_url":"https://codeload.github.com/vmariiechko/databricks-bundle-template/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vmariiechko%2Fdatabricks-bundle-template/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33765378,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-01T02:00:06.963Z","response_time":115,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ci-cd","dabs","data-engineering","databricks","declarative-automation-bundles","devops","template","unity-catalog"],"created_at":"2026-02-08T16:03:16.239Z","updated_at":"2026-06-01T07:32:39.896Z","avatar_url":"https://github.com/vmariiechko.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Databricks Multi-Environment Bundle Template\n\n[![Tests](https://github.com/vmariiechko/databricks-bundle-template/actions/workflows/test.yml/badge.svg)](https://github.com/vmariiechko/databricks-bundle-template/actions/workflows/test.yml)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n\nA custom template for [Declarative Automation Bundles](https://docs.databricks.com/aws/en/dev-tools/bundles/) that generates production-ready, multi-environment projects with configurable compute, permissions, and CI/CD pipelines.\n\n## Demo\n\n\u003cdiv align=\"center\"\u003e\n  \u003ca href=\"https://www.youtube.com/watch?v=-5PztHN3g70\"\u003e\n    \u003cimg src=\"https://img.youtube.com/vi/-5PztHN3g70/maxresdefault.jpg\" alt=\"Watch the video\" style=\"width:100%; max-width:600px;\"\u003e\n  \u003c/a\u003e\n\u003c/div\u003e\n\n## Why This Template?\n\nSetting up a production-grade Databricks project involves many decisions: environment isolation, compute configuration, RBAC permissions, service principal setup, CI/CD pipelines, and Unity Catalog schemas. This template encodes proven patterns for all of these so you can go from zero to a deployable bundle in minutes, not days.\n\n## Requirements\n\n- Databricks CLI v0.296.0+ (`pip install databricks-cli`)\n- Unity Catalog enabled workspace\n\n## Usage\n\n```bash\ndatabricks bundle init https://github.com/vmariiechko/databricks-bundle-template\n```\n\nOr from a local clone:\n\n```bash\ndatabricks bundle init /path/to/databricks-bundle-template\n```\n\nThe CLI will guide you through configuration options. Your generated project includes complete documentation for deployment and customization.\n\n**Windows users:** Use PowerShell or Command Prompt for interactive prompts. Git Bash is not supported for interactive mode.\n\n### Quick Start with Config File\n\nFor a quick, non-interactive setup or if you prefer to skip the prompts:\n\n```bash\n# From remote (create config.json with your values first):\ndatabricks bundle init https://github.com/vmariiechko/databricks-bundle-template \\\n  --config-file config.json\n```\n\n```bash\n# From local clone:\ncp tests/configs/full_with_sp.json my-config.json   # Copy and edit with your values\ndatabricks bundle init . --config-file my-config.json\n```\n\nSee [example configs](./tests/configs/) for options and refer to [Template Options](#template-options) for available values.\n\n## What You Get\n\n- **Multi-environment deployment** (user/stage/prod, optional dev)\n- **Unity Catalog integration** with medallion architecture schemas\n- **Sample ETL jobs and pipelines**\n- **Optional RBAC** with environment-aware group permissions\n- **Configurable compute** (classic clusters, serverless, or both)\n- **CI/CD pipeline templates** (Azure DevOps, GitHub Actions, GitLab)\n\nSee the [example project](https://github.com/vmariiechko/databricks-bundle-template-example) for a pre-generated output (AWS + GitHub Actions + classic compute, no RBAC).\n\n## Asset Library\n\nBeyond the main template, this repo ships an **asset library**: standalone sub-templates that install individual Databricks artifacts (recovery scripts, pipelines, jobs, dashboards) into any existing bundle, whether generated by this template or not. Each asset is self-contained and installed with a single CLI command.\n\nCurrent asset: **`sdp-checkpoint-recovery`** resets checkpoint selection on a Lakeflow Spark Declarative Pipeline after a source table has been dropped and recreated.\n\n```bash\ndatabricks bundle init https://github.com/vmariiechko/databricks-bundle-template \\\n  --template-dir assets/sdp-checkpoint-recovery\n```\n\nSee [ASSETS.md](ASSETS.md) for the full catalog and install pattern. Contributors: see [CONTRIBUTING.md — Adding an Asset](CONTRIBUTING.md#adding-an-asset).\n\n## Template Options\n\n| Option | Choices | Default |\n|--------|---------|---------|\n| Environment setup | `full` (user/stage/prod) / `minimal` (user/stage) | `full` |\n| Include dev environment | `yes` / `no` | `no` |\n| Compute type | `classic` / `serverless` / `both` | `classic` |\n| Cloud provider | `azure` / `aws` / `gcp` | `azure` |\n| Workspace setup | `single_workspace` / `multi_workspace` | `single_workspace` |\n| Include permissions | `yes` / `no` | `yes` |\n| Include CI/CD | `yes` / `no` | `yes` |\n| CI/CD platform | `azure_devops` / `github_actions` / `gitlab` | `azure_devops` |\n| Default branch | string | `main` |\n| Release branch | string (full mode only) | `release` |\n\n---\n\n## Template Development\n\nThis section is for developers modifying the template itself.\n\n### Repository Structure\n\n```\ndatabricks-bundle-template/\n├── databricks_template_schema.json   # Prompt definitions\n├── library/helpers.tmpl              # Go template helpers\n├── template/\n│   ├── update_layout.tmpl            # Conditional directory/file skipping\n│   └── {{.project_name}}/            # Generated project structure\n│       ├── .azure/                   # Azure DevOps CI/CD pipelines\n│       ├── .github/                  # GitHub Actions workflows\n│       ├── .gitlab-ci.yml.tmpl       # GitLab CI/CD pipeline\n│       ├── docs/                     # Setup guides (CI/CD, permissions, groups)\n│       └── ...                       # Bundle config, resources, src\n├── tests/                            # Pytest test suite\n├── ARCHITECTURE.md                   # Technical design\n└── DEVELOPMENT.md                    # Developer notes\n```\n\n### Testing\n\n```bash\n# Create and activate virtual environment\npython -m venv venv\nsource venv/bin/activate    # Linux/macOS\nvenv\\Scripts\\activate       # Windows\n\n# Install dependencies\npip install -r tests/requirements_dev.txt\n\n# Run automated tests\npytest tests/ -V\n```\n\n```bash\n# Manual testing\ndatabricks bundle init . --output-dir ../test-output --config-file tests/configs/full_with_dev.json\n\n# Validate (requires Databricks CLI authentication)\ncd ../test-output/test_full_with_dev/\ndatabricks bundle validate -t user\n```\n\n### Documentation\n\n- [ARCHITECTURE.md](ARCHITECTURE.md) - Design decisions and technical architecture\n- [DEVELOPMENT.md](DEVELOPMENT.md) - Development notes and testing matrix\n- [tests/README.md](tests/README.md) - Tests setup and guide\n\n---\n\n## Contributing\n\nContributions are welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines on how to get started.\n\n## Roadmap\n\nSee [ROADMAP.md](ROADMAP.md) for planned features and the project direction. Highlights include asset sub-templates and advanced permissions profiles.\n\n## Community\n\n- **Questions \u0026 help**: [GitHub Discussions](https://github.com/vmariiechko/databricks-bundle-template/discussions)\n- **Bug reports**: [Issue tracker](https://github.com/vmariiechko/databricks-bundle-template/issues)\n- **Feature ideas**: [Feature requests](https://github.com/vmariiechko/databricks-bundle-template/issues/new?template=2-feature-request.yml)\n\n## License\n\nThis project is licensed under the [MIT License](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvmariiechko%2Fdatabricks-bundle-template","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvmariiechko%2Fdatabricks-bundle-template","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvmariiechko%2Fdatabricks-bundle-template/lists"}