{"id":13487332,"url":"https://github.com/projectnessie/nessie","last_synced_at":"2026-02-04T11:50:39.669Z","repository":{"id":36967785,"uuid":"254450741","full_name":"projectnessie/nessie","owner":"projectnessie","description":"Nessie: Transactional Catalog for Data Lakes with Git-like semantics","archived":false,"fork":false,"pushed_at":"2025-04-18T01:51:51.000Z","size":194699,"stargazers_count":1182,"open_issues_count":125,"forks_count":148,"subscribers_count":28,"default_branch":"main","last_synced_at":"2025-04-18T08:54:31.970Z","etag":null,"topics":["aws-lambda","data","git","iceberg","java","spark"],"latest_commit_sha":null,"homepage":"https://projectnessie.org","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/projectnessie.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-04-09T18:39:03.000Z","updated_at":"2025-04-17T23:49:06.000Z","dependencies_parsed_at":"2023-09-23T05:14:34.985Z","dependency_job_id":"e62dabeb-6db0-487b-8486-8b0d401484a6","html_url":"https://github.com/projectnessie/nessie","commit_stats":{"total_commits":8440,"total_committers":65,"mean_commits":"129.84615384615384","dds":0.7136255924170616,"last_synced_commit":"1327fa941bc7875e278d2b2486b9b89206e5187a"},"previous_names":[],"tags_count":154,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/projectnessie%2Fnessie","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/projectnessie%2Fnessie/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/projectnessie%2Fnessie/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/projectnessie%2Fnessie/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/projectnessie","download_url":"https://codeload.github.com/projectnessie/nessie/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250513286,"owners_count":21443193,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws-lambda","data","git","iceberg","java","spark"],"created_at":"2024-07-31T18:00:57.824Z","updated_at":"2026-02-04T11:50:39.662Z","avatar_url":"https://github.com/projectnessie.png","language":"Java","funding_links":[],"categories":["Java","Data Lake Storages","Table of Contents","大数据","\u003ca name=\"Java\"\u003e\u003c/a\u003eJava","Open Source Tools","Data Lake Management"],"sub_categories":["Metadata Service","Catalog Implementations"],"readme":"# Project Nessie\n\nProject Nessie is a Transactional Catalog for Data Lakes with Git-like semantics.\n\n[![Zulip](https://img.shields.io/badge/Zulip-Chat-blue?color=3d4db3\u0026logo=zulip\u0026style=for-the-badge\u0026logoColor=white)](https://project-nessie.zulipchat.com/)\n[![Group Discussion](https://img.shields.io/badge/Discussion-Groups-blue.svg?color=3d4db3\u0026logo=google\u0026style=for-the-badge\u0026logoColor=white)](https://groups.google.com/g/projectnessie)\n[![Twitter](https://img.shields.io/badge/Twitter-Follow_Us-blue?color=3d4db3\u0026logo=twitter\u0026style=for-the-badge\u0026logoColor=white)](https://twitter.com/projectnessie)\n[![Website](https://img.shields.io/badge/https-projectnessie.org-blue?color=3d4db3\u0026logo=firefox\u0026style=for-the-badge\u0026logoColor=white)](https://projectnessie.org/)\n\n[![Maven Central](https://img.shields.io/maven-central/v/org.projectnessie.nessie/nessie?label=Maven%20Central\u0026logo=apachemaven\u0026color=3f6ec6\u0026style=for-the-badge\u0026logoColor=white)](https://search.maven.org/artifact/org.projectnessie.nessie/nessie)\n[![PyPI](https://img.shields.io/pypi/v/pynessie.svg?label=PyPI\u0026logo=python\u0026color=3f6ec6\u0026style=for-the-badge\u0026logoColor=white)](https://pypi.python.org/pypi/pynessie)\n[![quay.io Docker](https://img.shields.io/maven-central/v/org.projectnessie.nessie/nessie?label=quay.io+Docker\u0026logo=docker\u0026color=3f6ec6\u0026style=for-the-badge\u0026logoColor=white)](https://quay.io/repository/projectnessie/nessie?tab=tags)\n[![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/nessie\u0026color=3f6ec6\u0026labelColor=\u0026style=for-the-badge\u0026logoColor=white)](https://artifacthub.io/packages/search?repo=nessie)\n\n\n[![Build Status](https://img.shields.io/github/actions/workflow/status/projectnessie/nessie/ci.yml?branch=main\u0026label=Main%20CI\u0026logo=Github\u0026style=flat-square)](https://github.com/projectnessie/nessie/actions/workflows/ci.yml?query=branch%3Amain)\n[![Query Engines CI](https://img.shields.io/github/actions/workflow/status/projectnessie/query-engine-integration-tests/main.yml?label=Nessie%2FIceberg%20in-dev\u0026logo=Github\u0026style=flat-square)](https://github.com/projectnessie/query-engine-integration-tests/actions/workflows/main.yml?query=branch%3Amain)\n[![macOS Build](https://img.shields.io/github/actions/workflow/status/projectnessie/nessie/ci-mac.yml?label=macOS\u0026logo=apple\u0026style=flat-square)](https://github.com/projectnessie/nessie/actions/workflows/ci-mac.yml)\n\nMore information can be found at [projectnessie.org](https://projectnessie.org/).\n\nNessie supports Iceberg Tables/Views. Additionally, Nessie is focused on working with the widest range of tools possible, which can be seen in the [feature matrix](https://projectnessie.org/tools/#feature-matrix).\n\n## Using Nessie\n\nYou can quickly get started with Nessie by using our small, fast docker image.\n\n**IMPORTANT NOTE** Nessie has moved away from `docker.io` to GitHub's container registry `ghcr.io`,\nand also `quay.io`. Recent releases are already only available on both ghcr.io and quay.io. Please\nupdate references to `projectnessie/nessie` in your code to either `ghcr.io/projectnessie/nessie`\nor `quay.io/projectnessie/nessie`.\n\n```\ndocker pull ghcr.io/projectnessie/nessie\ndocker run -p 19120:19120 ghcr.io/projectnessie/nessie\n```\n_For trying Nessie image with different configuration options, refer to the templates under the [docker module](./docker#readme)._\u003cbr\u003e\n\nA local [Web UI](https://projectnessie.org/tools/ui/) will be available at this point.\n\nThen install the Nessie CLI tool (to learn more about CLI tool and how to use it, check [Nessie CLI Documentation](https://projectnessie.org/tools/cli/)).\n\n```\npip install pynessie\n```\n\nFrom there, you can use one of our technology integrations such those for \n\n* [Spark via Iceberg](https://projectnessie.org/tools/iceberg/spark/)\n* [Hive via Iceberg](https://projectnessie.org/tools/iceberg/hive/)\n\nTo learn more about all supported integrations and tools, check [here](https://projectnessie.org/tools/) \n\nHave fun! We have a Google Group and a Slack channel we use for both developers and \nusers. Check them out [here](https://projectnessie.org/community/).\n\n### Authentication\n\nBy default, Nessie servers run with authentication disabled and all requests are processed under the \"anonymous\"\nuser identity.\n\nNessie supports bearer tokens and uses [OpenID Connect](https://openid.net/connect/) for validating them.\n\nAuthentication can be enabled by setting the following Quarkus properties:\n* `nessie.server.authentication.enabled=true`\n* `quarkus.oidc.auth-server-url=\u003cOpenID Server URL\u003e`\n* `quarkus.oidc.client-id=\u003cClient ID\u003e`\n\n#### Experimenting with Nessie Authentication in Docker\n\nOne can start the `projectnessie/nessie` docker image in authenticated mode by setting\nthe properties mentioned above via docker environment variables. For example:\n\n```shell\ndocker run -p 19120:19120 \\\n  -e QUARKUS_OIDC_CLIENT_ID=\u003cClient ID\u003e \\\n  -e QUARKUS_OIDC_AUTH_SERVER_URL=\u003cOpenID Server URL\u003e \\\n  -e NESSIE_SERVER_AUTHENTICATION_ENABLED=true \\\n  --network host \\\n  ghcr.io/projectnessie/nessie\n```\n\n## Building and Developing Nessie\n\n### Requirements\n\n- JDK 21 or higher: JDK 21 or higher is needed to build Nessie (some artifacts are built \n  for Java 8 or 11)\n\n### Installation\n\nClone this repository:\n```bash\ngit clone https://github.com/projectnessie/nessie\ncd nessie\n```\n\nThen open the project in IntelliJ or Eclipse, or just use the IDEs to clone this github repository.\n\nRefer to [CONTRIBUTING](./CONTRIBUTING.md) for build instructions.\n\n### Compatibility\n\nNessie Iceberg's integration is compatible with Iceberg as in the following table:\n\n| Nessie version | Iceberg version | Spark version (Scala 2.12+2.13) | Hive version | Flink version          | Presto version                      | Trino version |\n|----------------|-----------------|---------------------------------|--------------|------------------------|-------------------------------------|---------------|\n| 0.107.1         | 1.5.0           | 3.3.x, 3.4.x, 3.5.x             | n/a          | 1.16.x, 1.17.x, 1.18.x | 0.277, 0.278.x, 0.279, 0.280, 0.281 | 419           |\n\n### Distribution\nTo run:\n1. configuration in `servers/quarkus-server/src/main/resources/application.properties`\n2. execute `./gradlew :nessie-quarkus:assemble \u0026\u0026 java -jar servers/quarkus-server/build/quarkus-app/quarkus-run.jar`\n3. go to `http://localhost:19120`\n\n### UI \n\nNessie UI sources have moved to their own repository: https://github.com/projectnessie/nessie-ui.\n\n### Docker image\n\nOfficial Nessie images are built with support for [multiplatform builds](./tools/dockerbuild#readme). But to quickly\nbuild a docker image for testing purposes, simply run the following command:\n\n```shell\n./gradlew :nessie-quarkus:clean :nessie-quarkus:quarkusBuild\ndocker build -f ./tools/dockerbuild/docker/Dockerfile-server -t nessie-unstable:latest ./servers/quarkus-server \n```\n\nCheck that your image is available locally:\n\n```shell\ndocker images\n```\n\nYou should see something like this:\n\n```\nREPOSITORY       TAG     IMAGE ID       CREATED          SIZE\nnessie-unstable  latest  24bb4c7bd696   15 seconds ago   555MB\n```\n\nOnce this is done you can run your image with `docker run -p 19120:19120 quay.io/nessie-unstable:latest`, passing the relevant\nenvironment variables, if any. Environment variables names must follow MicroProfile Config's [mapping\nrules](https://github.com/eclipse/microprofile-config/blob/master/spec/src/main/asciidoc/configsources.asciidoc#environment-variables-mapping-rules).\n\n## Nessie related repositories\n\n* [CEL Java](https://github.com/projectnessie/cel-java): Java port of the Common Expression Language\n* [Nessie apprunner](https://github.com/projectnessie/nessie-apprunner): Maven and Gradle plugins to use Nessie in integration tests.\n\n## Contributing\n\n### Code Style\n\nThe Nessie project uses the Google Java Code Style, scalafmt and pep8.\nSee [CONTRIBUTING.md](./CONTRIBUTING.md) for more information.\n\n## Acknowledgements\n\nSee [ACKNOWLEDGEMENTS.md](ACKNOWLEDGEMENTS.md)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprojectnessie%2Fnessie","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprojectnessie%2Fnessie","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprojectnessie%2Fnessie/lists"}