{"id":21262866,"url":"https://github.com/stackql/stackql-server","last_synced_at":"2025-03-15T07:44:03.115Z","repository":{"id":208919417,"uuid":"722738963","full_name":"stackql/stackql-server","owner":"stackql","description":null,"archived":false,"fork":false,"pushed_at":"2024-04-06T22:02:49.000Z","size":37,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-21T22:43:07.556Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stackql.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-23T20:42:57.000Z","updated_at":"2024-01-08T01:38:48.000Z","dependencies_parsed_at":"2024-01-14T22:45:01.427Z","dependency_job_id":"7bde5e01-d615-43f9-81d0-f8503a69dc5d","html_url":"https://github.com/stackql/stackql-server","commit_stats":null,"previous_names":["stackql/stackql-server"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stackql%2Fstackql-server","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stackql%2Fstackql-server/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stackql%2Fstackql-server/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stackql%2Fstackql-server/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stackql","download_url":"https://codeload.github.com/stackql/stackql-server/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243701314,"owners_count":20333616,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-21T04:59:34.197Z","updated_at":"2025-03-15T07:44:03.093Z","avatar_url":"https://github.com/stackql.png","language":"Shell","readme":"# StackQL Server with PostgreSQL Backend\n\n# Table of Contents\n\n1. [Architecture](#architecture)\n2. [Deployment Options](#deployment-options)\n   - [Container Deployment Modes](#container-deployment-modes)\n   - [Database Configuration Modes](#database-configuration-modes)\n   - [Client/Server Authentication Modes](#clientserver-authentication-modes)\n3. [Authenticating to Cloud Providers](#authenticating-to-cloud-providers)\n4. [Building the Container](#building-the-container)\n4. [Running the Container](#running-the-container)\n   - [Without mTLS (`SECURE_MODE=false`)](#without-mtls-secure_modefalse)\n   - [With mTLS (`SECURE_MODE=true`)](#with-mtls-secure_modetrue)\n5. [Running the Container in Azure Container Instances (ACI)](#running-the-container-in-azure-container-instances-aci)\n\n## Architecture\n\nThe architecture consists of two primary components:\n\n1. **StackQL Server**: A server that starts a [StackQL](https://github.com/stackql/stackql) server, accepting StackQL queries using the PostgreSQL wire protocol.\n2. **PostgreSQL Server**: A backend database server used for relational algebra and temporary storage, particularly for materialized views.\n\n```mermaid\nflowchart TD;\nsubgraph Docker_or_ACI[\"Docker or Azure Container Instances (ACI)\"];\nB[StackQL Server];\nC[\"Local PostgreSQL Instance\\n(if POSTGRES_HOST == 127.0.0.1)\"];\nB \u003c-- uses --\u003e C;\nend;\nA[StackQL Client] \u003c-- uses --\u003e B;\nB \u003c-- gets data from\\nor interacts with --\u003e E[Cloud/SaaS Providers];\n%% KV[Azure Key Vault] -.-\u003e|\"Stores Secrets\\nfor SECURE_MODE\\n(if KEYVAULT_NAME \u0026\u0026 KEYVAULT_CREDENTIAL)\"| B;\nB \u003c-.-\u003e|if POSTGRES_HOST != 127.0.0.1| RemoteDB[\"Remote PostgreSQL Database\"];\n```\n\n## Deployment Options\n\nThe different deployment options are as follows:\n\n### Container Deployment Modes\n- **Deployment via ACI**: Leverages Azure Container Instances for scalable, cloud-native deployments.\n- **Deployed using `docker run`**: Ideal for containerized environments, ensuring consistency and portability across different systems.\n\n### Database Configuration Modes\n- **Local DB Mode**: \n  - Activated when `POSTGRES_HOST` is set to `127.0.0.1` (default).\n  - Runs a local, embedded PostgreSQL backend database.\n- **Remote DB Mode**: \n  - Triggered when `POSTGRES_HOST` is set to any value other than `127.0.0.1`.\n  - Connects to an externally hosted PostgreSQL database.\n\n### Client/Server Authentication Modes\n- **mTLS Authentication**:\n  - Enabled by setting `SECURE_MODE=true` (default is `false`).\n  - Utilizes mutual TLS (mTLS) for enhanced security in communications.\n- **Keys and Certificates**:\n  - Can be directly copied into the container.\n  - Alternatively, sourced from Azure Key Vault if `KEYVAULT_NAME` and `KEYVAULT_CREDENTIAL` are provided.\n\n## Authenticating to Cloud Providers\n\nPopulate the necessary environment variables to authenticate with your specific cloud providers. For more information on which environment variables to populate, see the [StackQL provider registry](https://github.com/stackql/stackql-provider-registry) documentation.\n\n## Building the Container\n\nTo build the container, run the following command:\n\n```bash\ndocker build --no-cache -t stackql-server .\n```\n\n## Running the Container\n\n### Without mTLS (`SECURE_MODE=false`)\n\nTo run the container locally without mTLS, use the following command:\n\n```bash\n# Use -e to supply provider credentials as needed (GitHub credentials used in this example)\ndocker run -d -p 7432:7432 \\\n-e STACKQL_GITHUB_USERNAME \\\n-e STACKQL_GITHUB_PASSWORD \\\nstackql-server\n\n# or if using the Dockerhub image...\ndocker run -d -p 7432:7432 \\\n-e STACKQL_GITHUB_USERNAME \\\n-e STACKQL_GITHUB_PASSWORD \\\nstackql/stackql-server\n```\n\nTo connect to the server (not configured for mTLS), use the following command:\n\n```bash\nexport PGSSLMODE=allow # or disable\npsql -h localhost -p 7432 -U stackql -d stackql\n```\n\nTo stop the container, use the following command:\n\n```bash\ndocker stop $(docker ps -a -q --filter ancestor=stackql-server)\n\n# or if using the Dockerhub image...\n\ndocker stop $(docker ps -a -q --filter ancestor=stackql/stackql-server)\n```\n\n### With mTLS (`SECURE_MODE=true`)\n\nTo prepare certificates and keys, run the following commands:\n\n```bash\n# Follow these steps to generate Root CA, Server Cert, and Client Cert\nopenssl req -x509 -keyout creds/server_key.pem -out creds/server_cert.pem -config creds/openssl.cnf -days 365\nopenssl req -x509 -keyout creds/client_key.pem -out creds/client_cert.pem -config creds/openssl.cnf -days 365\nchmod 400 creds/client_key.pem\n```\n\nTo run the container locally with mTLS, use the following command:\n\n```bash\ndocker run -d -p 7432:7432 \\\n-e STACKQL_GITHUB_USERNAME \\\n-e STACKQL_GITHUB_PASSWORD \\\n-e SECURE_MODE=true -v $(pwd)/creds:/opt/stackql/srv/credentials \\\nstackql-server\n\n# or if using the Dockerhub image...\n\ndocker run -d -p 7432:7432 \\\n-e STACKQL_GITHUB_USERNAME \\\n-e STACKQL_GITHUB_PASSWORD \\\n-e SECURE_MODE=true -v $(pwd)/creds:/opt/stackql/srv/credentials \\\nstackql/stackql-server\n```\n\nTo connect to the server (configured for mTLS), use the following command:\n\n```bash\nPGSSLCERT=creds/client_cert.pem\nPGSSLKEY=creds/client_key.pem\nPGSSLROOTCERT=creds/server_cert.pem\nPGSSLMODE=require\npsql -h localhost -p 7432 -d stackql\n```\n\n## Running the Container in Azure Container Instances (ACI)\n\nTo deploy the container in Azure Container Instances (ACI) using an image from Docker Hub, you can follow these steps:\n\n1. **Create an Azure Container Instance:**\nTo create an instance, use the Azure CLI. Replace values for `name`, `resource-group`, and `dns-name-label` with your specific details. The `--dns-name-label` should be a unique DNS name for the ACI.\n```bash\nSERVER_CERT=$(base64 -w 0 creds/server_cert.pem)\nSERVER_KEY=$(base64 -w 0 creds/server_key.pem)\nCLIENT_CERT=$(base64 -w 0 creds/client_cert.pem)\n\naz container create \\\n--name stackqlserver \\\n--resource-group stackql-activity-monitor-rg \\\n--image docker.io/stackql/stackql-server:latest \\\n--dns-name-label stackql \\\n--ports 7432 \\\n--protocol TCP \\\n--environment-variables \\\nSECURE_MODE=true \\\nSERVER_CERT=$SERVER_CERT \\\nSERVER_KEY=$SERVER_KEY \\\nCLIENT_CERT=$CLIENT_CERT\n```\nReplace the environment variable values with the ones you need for your setup.\n\n2. **Retrieve the Fully Qualified Domain Name (FQDN) of the ACI:**\nAfter the ACI is successfully deployed, retrieve its FQDN:\n```bash\naz container show \\\n--name stackqlserver \\\n--resource-group stackql-activity-monitor-rg \\\n--query ipAddress.fqdn \\\n--output tsv\n```\n\n4. **Connect to the Server:**\nUse the FQDN obtained above to connect to your StackQL server using a PostgreSQL client.  For connections over mTLS, ensure that the client machine has the necessary client certificates configured, for example:\n\n```bash\nexport PGSSLCERT=creds/client_cert.pem\nexport PGSSLKEY=creds/client_key.pem\nexport PGSSLROOTCERT=creds/server_cert.pem\nexport PGSSLMODE=require\npsql -h stackql.eastus.azurecontainer.io -p 7432 -d stackql\n```\n\n5. **Monitor the Container Instance:**\n\nTo quickly check the logs of your container instance, you can use the Azure CLI:\n```bash\naz container logs \\\n--resource-group stackql-activity-monitor-rg \\\n--name stackqlserver\n```\nThis command retrieves the logs produced by the container.\n\n6. **Delete the Container Instance:**\n\nTo delete the container instance, use the Azure CLI:\n```bash\naz container delete \\\n--resource-group stackql-activity-monitor-rg \\\n--name stackqlserver\n```","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstackql%2Fstackql-server","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstackql%2Fstackql-server","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstackql%2Fstackql-server/lists"}