{"id":23524191,"url":"https://github.com/kimtth/bicep-azure-data-platform-lac","last_synced_at":"2026-01-22T08:32:57.189Z","repository":{"id":103354652,"uuid":"514572421","full_name":"kimtth/bicep-azure-data-platform-laC","owner":"kimtth","description":"🗄️ 👨🏾‍💻🏭Azure Data platform Infrastructure as Code (Datafactory, Databricks, Synapse Analytics, Purview)","archived":false,"fork":false,"pushed_at":"2022-07-17T14:44:32.000Z","size":55,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-14T06:27:27.644Z","etag":null,"topics":["bicep","data-platform","infrastructure-as-code"],"latest_commit_sha":null,"homepage":"","language":"Bicep","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kimtth.png","metadata":{"files":{"readme":"Readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-07-16T12:28:51.000Z","updated_at":"2025-04-11T15:03:12.000Z","dependencies_parsed_at":"2023-07-07T21:15:16.102Z","dependency_job_id":null,"html_url":"https://github.com/kimtth/bicep-azure-data-platform-laC","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/kimtth/bicep-azure-data-platform-laC","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kimtth%2Fbicep-azure-data-platform-laC","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kimtth%2Fbicep-azure-data-platform-laC/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kimtth%2Fbicep-azure-data-platform-laC/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kimtth%2Fbicep-azure-data-platform-laC/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kimtth","download_url":"https://codeload.github.com/kimtth/bicep-azure-data-platform-laC/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kimtth%2Fbicep-azure-data-platform-laC/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28659518,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-22T01:17:37.254Z","status":"online","status_checked_at":"2026-01-22T02:00:07.137Z","response_time":144,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bicep","data-platform","infrastructure-as-code"],"created_at":"2024-12-25T18:14:50.503Z","updated_at":"2026-01-22T08:32:57.184Z","avatar_url":"https://github.com/kimtth.png","language":"Bicep","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# Intrastructure as a code : Bicep - Azure Data Platforms\n\nIn this chapter, we are going to deploy Azure Data Platforms by Bicep. Infrastructure automation reduces redundant manipulations. By using this code we can deploy all data platforms within approx. 3-4 mins.\n\n- What is Bicep?\n\nBicep is a domain-specific language (DSL) that uses declarative syntax to deploy Azure resources. \n\nUnder the hood, Bicep (DSL) is converted into JSON which would be passed into Azure Resource Manager. \n\n- Bicep (DSL) -\u003e JSON -\u003e Azure Resource Manager -\u003e Azure\n\n# Azure Data services overview\n\nMicrosoft Azure offers services for a wide variety of data-related needs. \n\n- Data factory: Data pipelines. Integration of data sources, ETL, and data flows\n- Synapse Analytics: Data lake and Datawarhouse platform. T-SQL Polybase and Spark\n- Databricks: PySpark-based cloud service\n- Purview: Data Governance solution to track and monitor, an overview of data lineage\n- Azure HDInsight: an open-source analytics service that runs Hadoop, Spark, Kafka, and more (Most of cases, HDInsight can be replaced by the other azure services)\n\n# Databricks vs Synapse Analytics\n\nSynapse has an open-source Spark version with built-in support for . NET, whereas Databricks has an optimized version of Spark which offers increased performance.\n\nhttps://adatis.co.uk/databricks-vs-synapse-spark-pools-what-when-and-where/\n\n| Azure             | AWS               | GCP               |\n|-------------------|-------------------|-------------------|\n| Data factory      | Glue              | Cloud DataPrep    |\n| Synapse Analytics | Redshift          | BigQuery          |\n| Databricks        | Databricks on AWS | Databricks on GCP |\n| Purview           | -                 | -                 |\n\n# Azure DL/DW model architecutre \n\n ![mda](img/modern-data-architecture.png \"data architecture\")\n\n https://docs.microsoft.com/en-in/azure/architecture/solution-ideas/articles/enterprise-data-warehouse\n\n# Bicep structure\n\n```\nproject\n│   README.md\n│   init.ps1    \n│   main.bicep\n└───img\n│   └───...\n└───template\n    │   databricks.bicep\n    │   datafactory.bicep\n    │   purview.bicep\n    └── synapse.bicep\n```\n\n```powershell\nPS\u003e az group create --name $resourceGroup --location $location\nPS\u003e az deployment group create --resource-group $resourceGroup --template-file .\\main.bicep --parameters .\\deploy.parameter.json\n```\n\n# How to execute the script\n\n- Execution flow\n\n```cmd\n.\\init.ps1 -resourceGroup bicepRG -location japaneast\n```\n\ninit.ps1 --\u003e main.bicep --\u003e execute in sequence `bicep` files in template directory\n\n- init.ps1: Pass parameters and biceps to Azure CLI\n- main.bicep: Consolidate the bicep files and control modules dependencies \n- template/\u003cservice_name\u003e.bicep: Define Each service's specifications\n\n# References\n\n- [Bicep overview](https://docs.microsoft.com/en-us/azure/azure-resource-manager/bicep/overview?tabs=bicep)\n\n- [Bicep CLI](https://docs.microsoft.com/en-us/azure/azure-resource-manager/bicep/deploy-cli)\n\n- [Azure Bicep/ARM template](https://docs.microsoft.com/en-us/azure/templates/)\n\n## Bicep Anti pattern - dependsOn syntax\n\n```\nresource dnsZone 'Microsoft.Network/dnszones@2018-05-01' = {\n  name: 'demoeZone1'\n  location: 'global'\n}\n\nresource otherZone 'Microsoft.Network/dnszones@2018-05-01' = {\n  name: 'demoZone2'\n  location: 'global'\n  dependsOn: [\n    dnsZone\n  ]\n}\n```\n\nAzure Resource Manager evaluates the dependencies between resources, and deploys them in their dependent order. When resources aren't dependent on each other, Resource Manager deploys them in parallel.\n\nThe use case for dependsOn is if you need to make sure that the Resource Manager will create the resource with the dependOn only after the other resource has been created (and not in parallel).\n\nhttps://stackoverflow.com/questions/71320257/why-is-dependson-not-recommended-in-bicep\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkimtth%2Fbicep-azure-data-platform-lac","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkimtth%2Fbicep-azure-data-platform-lac","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkimtth%2Fbicep-azure-data-platform-lac/lists"}