{"id":29570037,"url":"https://github.com/finos/legend-community-delta","last_synced_at":"2025-10-15T14:09:47.442Z","repository":{"id":40491981,"uuid":"375358812","full_name":"finos/legend-community-delta","owner":"finos","description":"Combining best of open data standards with open source technologies","archived":false,"fork":false,"pushed_at":"2023-04-24T15:01:52.000Z","size":2241,"stargazers_count":8,"open_issues_count":7,"forks_count":6,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-10-11T00:24:48.906Z","etag":null,"topics":["data","database","delta-lake","spark"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/finos.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-06-09T13:07:56.000Z","updated_at":"2025-09-27T15:33:36.000Z","dependencies_parsed_at":"2024-11-15T12:33:48.092Z","dependency_job_id":null,"html_url":"https://github.com/finos/legend-community-delta","commit_stats":null,"previous_names":["finos/legend-delta"],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/finos/legend-community-delta","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/finos%2Flegend-community-delta","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/finos%2Flegend-community-delta/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/finos%2Flegend-community-delta/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/finos%2Flegend-community-delta/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/finos","download_url":"https://codeload.github.com/finos/legend-community-delta/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/finos%2Flegend-community-delta/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279014026,"owners_count":26085345,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-13T02:00:06.723Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data","database","delta-lake","spark"],"created_at":"2025-07-19T02:07:57.474Z","updated_at":"2025-10-15T14:09:47.414Z","avatar_url":"https://github.com/finos.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Legend on Delta Lake\n\n*An extension to the [Legend](https://legend.finos.org/) framework for spark / delta lake based environment, combining\nbest of open data standards with open source technologies*\n\n[![FINOS - Incubating](https://cdn.jsdelivr.net/gh/finos/contrib-toolbox@master/images/badge-incubating.svg)](https://finosfoundation.atlassian.net/wiki/display/FINOS/Incubating)\n[![Build CI](https://github.com/finos/legend-engine/workflows/Build%20CI/badge.svg)]()\n[![Maven Central](https://img.shields.io/maven-central/v/org.finos.legend-community/legend-delta.svg)](http://search.maven.org/#search%7Cga%7C1%7Ca%3A%22legend-delta)\n\n___\n\nIn addition to the JDBC connectivity enabled to Databricks from the \n[legend-engine](https://github.com/finos/legend-engine/tree/master/docs/databricks) itself, this project helps \norganizations define data models that can be converted into efficient data pipelines, ensuring data being queried\nis of high quality and availability. Raw data can be ingested as stream or batch and processed in line with the \nbusiness semantics defined from the Legend interface. Domain specific language defined in Legend Studio can be \ninterpreted as a series of Spark SQL operations, helping analysts create [Delta Lake](https://delta.io/) tables that \nnot only guarantees schema definition but also complies with expectations, derivations and constraints defined by \nbusiness analysts.\n\n\u003cimg src=\"https://raw.githubusercontent.com/finos/legend-community-delta/main/images/legend-delta-workflow.png\" width=\"500\"\u003e\n\n## Usage\n\nMake sure to have the jar file of `org.finos.legend-community:legend-delta:X.Y.Z` and all its dependencies available in\nyour spark classpath and a legend data model (version controlled on gitlab) previously compiled to disk or packaged\nas a jar file and available in your classpath. For python support, please add the corresponding library from pypi\n[repo](https://pypi.org/project/legend-delta/).\n\n```shell\npip install legend-delta==X.Y.Z\n```\n\nWe show you how to extract schema, retrieve and enforce expectations and create delta tables in both\n[scala](https://github.com/finos/legend-delta/blob/main/databricks-scala.ipynb) and \n[python](https://github.com/finos/legend-delta/blob/main/databricks-python.ipynb) sample notebooks.\n\n## Author\n\nDatabricks, Inc.\n\n## License\n\nCopyright 2021 Databricks, Inc.\n\nDistributed under the [Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0).\n\nSPDX-License-Identifier: [Apache-2.0](https://spdx.org/licenses/Apache-2.0)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffinos%2Flegend-community-delta","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffinos%2Flegend-community-delta","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffinos%2Flegend-community-delta/lists"}