{"id":19705530,"url":"https://github.com/treeverse/lakefs-iceberg","last_synced_at":"2026-06-13T17:01:57.334Z","repository":{"id":177283528,"uuid":"645817023","full_name":"treeverse/lakefs-iceberg","owner":"treeverse","description":"A custom Iceberg catalog implementation for lakeFS","archived":false,"fork":false,"pushed_at":"2023-11-07T14:27:41.000Z","size":826,"stargazers_count":1,"open_issues_count":5,"forks_count":1,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-02-27T17:31:36.210Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/treeverse.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-26T13:57:55.000Z","updated_at":"2023-06-29T12:17:40.000Z","dependencies_parsed_at":"2024-11-11T21:29:46.055Z","dependency_job_id":"363c3d13-b4a1-47d6-b0c7-37af15c35d42","html_url":"https://github.com/treeverse/lakefs-iceberg","commit_stats":null,"previous_names":["treeverse/lakefs-iceberg"],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/treeverse/lakefs-iceberg","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/treeverse%2Flakefs-iceberg","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/treeverse%2Flakefs-iceberg/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/treeverse%2Flakefs-iceberg/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/treeverse%2Flakefs-iceberg/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/treeverse","download_url":"https://codeload.github.com/treeverse/lakefs-iceberg/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/treeverse%2Flakefs-iceberg/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34292326,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-13T02:00:06.617Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T21:28:48.745Z","updated_at":"2026-06-13T17:01:57.293Z","avatar_url":"https://github.com/treeverse.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg src=\"https://docs.lakefs.io/assets/logo.svg\" alt=\"lakeFS logo\" width=300/\u003e\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u003cimg src=\"https://www.apache.org/logos/res/iceberg/iceberg.png\" alt=\"Apache Iceberg logo\" width=300/\u003e\n\n## lakeFS Iceberg Catalog\n\nlakeFS enriches your Iceberg tables with Git capabilities: create a branch and make your changes in isolation, without affecting other team members.\n\nSee the instructions below on how to use it, and check out the integration in action in the [lakeFS samples repository](https://github.com/treeverse/lakeFS-samples/).\n\n### Install\n\nUse the following Maven dependency to install the lakeFS custom catalog:\n\n```xml\n\u003cdependency\u003e\n  \u003cgroupId\u003eio.lakefs\u003c/groupId\u003e\n  \u003cartifactId\u003elakefs-iceberg\u003c/artifactId\u003e\n  \u003cversion\u003e0.1.4\u003c/version\u003e\n\u003c/dependency\u003e\n```\n\n### Configure\n\nHere is how to configure the lakeFS custom catalog in Spark:\n```scala\nconf.set(\"spark.sql.catalog.lakefs\", \"org.apache.iceberg.spark.SparkCatalog\");\nconf.set(\"spark.sql.catalog.lakefs.catalog-impl\", \"io.lakefs.iceberg.LakeFSCatalog\");\nconf.set(\"spark.sql.catalog.lakefs.warehouse\", \"lakefs://example-repo\");\n```\n\nYou will also need to configure the S3A Hadoop FileSystem to interact with lakeFS:\n```scala\nconf.set(\"fs.s3a.access.key\", \"AKIAlakefs12345EXAMPLE\")\nconf.set(\"fs.s3a.secret.key\", \"abc/lakefs/1234567bPxRfiCYEXAMPLEKEY\")\nconf.set(\"fs.s3a.endpoint\", \"https://example-org.us-east-1.lakefscloud.io\")\nconf.set(\"fs.s3a.path.style.access\", \"true\")\n```\n\n### Create a table\n\nTo create a table on your main branch, use the following syntax:\n\n```sql\nCREATE TABLE lakefs.main.table1 (id int, data string);\n```\n\n### Create a branch\n\nWe can now commit the creation of the table to the main branch:\n\n```\nlakectl commit lakefs://example-repo/main -m \"my first iceberg commit\"\n```\n\nThen, create a branch:\n\n```\nlakectl branch create lakefs://example-repo/dev -s lakefs://example-repo/main\n```\n\n### Make changes on the branch\n\nWe can now make changes on the branch:\n\n```sql\nINSERT INTO lakefs.dev.table1 VALUES (3, 'data3');\n```\n\n### Query the table\n\nIf we query the table on the branch, we will see the data we inserted:\n\n```sql\nSELECT * FROM lakefs.dev.table1;\n```\n\nResults in:\n```\n+----+------+\n| id | data |\n+----+------+\n| 1  | data1|\n| 2  | data2|\n| 3  | data3|\n+----+------+\n```\n\nHowever, if we query the table on the main branch, we will not see the new changes:\n\n```sql\nSELECT * FROM lakefs.main.table1;\n```\n\nResults in:\n```\n+----+------+\n| id | data |\n+----+------+\n| 1  | data1|\n| 2  | data2|\n+----+------+\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftreeverse%2Flakefs-iceberg","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftreeverse%2Flakefs-iceberg","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftreeverse%2Flakefs-iceberg/lists"}