https://github.com/apache/polaris
Apache Polaris, the interoperable, open source catalog for Apache Iceberg
https://github.com/apache/polaris
apache iceberg polaris
Last synced: about 2 months ago
JSON representation
Apache Polaris, the interoperable, open source catalog for Apache Iceberg
- Host: GitHub
- URL: https://github.com/apache/polaris
- Owner: apache
- License: apache-2.0
- Created: 2024-05-29T18:44:27.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2025-04-09T22:08:06.000Z (about 2 months ago)
- Last Synced: 2025-04-10T11:59:12.927Z (about 2 months ago)
- Topics: apache, iceberg, polaris
- Language: Java
- Homepage: https://polaris.apache.org/
- Size: 10.9 MB
- Stars: 1,415
- Watchers: 90
- Forks: 218
- Open Issues: 183
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Codeowners: .github/CODEOWNERS
- Security: SECURITY.md
Awesome Lists containing this project
README
# Apache Polaris (incubating)
Apache Polaris™ is an open-source, fully-featured catalog for Apache Iceberg™. It implements Iceberg's
[REST API](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml),
enabling seamless multi-engine interoperability across a wide range of platforms, including Apache Doris™, Apache Flink®,
Apache Spark™, StarRocks, and Trino.
Documentation is available at https://polaris.apache.org, including
[Polaris management API doc](https://polaris.apache.org/index.html#tag/polaris-management-service_other)
and [Apache Iceberg REST API doc](https://polaris.apache.org/index.html#tag/Configuration-API).
[Subscribe to the dev mailing list][dev-list-subscribe] to join discussions via email or browse [the archives](https://lists.apache.org/[email protected]). Check out the [CONTRIBUTING guide](CONTRIBUTING.md)
for contribution guidelines.
[](https://join.slack.com/t/apache-polaris/shared_invite/zt-2y3l3r0fr-VtoW42ltir~nSzCYOrQgfw)
[](https://github.com/apache/polaris/actions/workflows/gradle.yml?query=branch%3Amain)
[dev-list-subscribe]: mailto:[email protected]
## Building and Running
Apache Polaris is organized into the following modules:
- `polaris-core` - The main Polaris entity definitions and core business logic
- API modules (implementing the Iceberg REST API and Polaris management API):
- `polaris-api-management-model` - The Polaris management model
- `polaris-api-management-service` - The Polaris management service
- `polaris-api-iceberg-service` - The Iceberg REST service
- Service modules:
- `polaris-service-common` - The main components of the Polaris server
- Quarkus runtime modules:
- `polaris-quarkus-service` - The Quarkus-specific components of the Polaris server
- `polaris-quarkus-defaults` - The Quarkus-specific configuration defaults
- `polaris-quarkus-server` - The Polaris server runtime
- `polaris-quarkus-admin-tool` - The Polaris admin & maintenance tool
- Persistence modules:
- `polaris-jpa-model` - The JPA entity definitions
- `polaris-eclipselink` - The Eclipselink implementation of the MetaStoreManager interface
Apache Polaris is built using Gradle with Java 21+ and Docker 27+.
- `./gradlew build` - To build and run tests. Make sure Docker is running, as the integration tests depend on it.
- `./gradlew assemble` - To skip tests.
- `./gradlew test` - To run unit tests and integration tests.
- `./gradlew run` - To run the Polaris server locally; the server is reachable at
localhost:8181. This is also suitable for running regression tests, or for connecting with Spark.
- `./regtests/run_spark_sql.sh` - To connect from Spark SQL. Here are some example commands to run in the Spark SQL shell:
```sql
create database db1;
show databases;
create table db1.table1 (id int, name string);
insert into db1.table1 values (1, 'a');
select * from db1.table1;
```
- `env POLARIS_HOST=localhost ./regtests/run.sh` - To run regression tests locally, see more options [here](./regtests/README.md).
### More build and run options
#### Running in Docker
- `./gradlew clean :polaris-quarkus-server:assemble -Dquarkus.container-image.build=true --no-build-cache` - To
build the image locally.
- `docker run -p 8181:8181 -p 8182:8182 apache/polaris:latest` - To run the image.
The Polaris codebase contains some docker compose examples to quickly get started with Polaris,
using different configurations. Check the `./getting-started` directory for more information.
#### Running in Kubernetes
- `./run.sh` - To run Polaris as a mini-deployment locally. This will create a Kind cluster,
then deploy one pod and one service. The service is available on ports `8181` and `8182`.
- `kubectl port-forward svc/polaris-service -n polaris 8181:8181 8182:8182` - To create secure
connections between a local machine and a pod within the cluster for both service and
health/metrics endpoints:
- http://localhost:8182/q/metrics
- http://localhost:8182/q/health
- `kubectl get pods -n polaris` - To check the status of the pods.
- `kubectl get deployment -n polaris` - To check the status of the deployment.
- `kubectl describe deployment polaris-deployment -n polaris` - To troubleshoot if things aren't working as expected.
#### Configuring Polaris
Polaris Servers can be configured using a variety of ways.
Please see the [Configuration Guide](site/content/in-dev/unreleased/configuration.md)
for more information.
Default configuration values can be found in `quarkus/defaults/src/main/resources/application.properties`.
#### Building docs
- Docs are generated using [Hugo](https://gohugo.io/) using the [Docsy](https://www.docsy.dev/docs/) theme.
- To view the site locally, run
```bash
site/bin/run-hugo-in-docker.sh
```
- See [README in `site/`](site/README.md) for more information.
## License
Apache Polaris is under the Apache License Version 2.0. See the [LICENSE](LICENSE).
## ASF Incubator disclaimer
Apache Polaris™ is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.
Apache®, Apache Polaris™, Apache Iceberg™, Apache Spark™ are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries.