{"id":16543330,"url":"https://github.com/codemation/pyql-cluster","last_synced_at":"2025-07-01T14:36:39.428Z","repository":{"id":39831524,"uuid":"221682219","full_name":"codemation/pyql-cluster","owner":"codemation","description":"Manager for multiple pyql-rest endpoints to allow for read-replicas \u0026 db mirroring","archived":false,"fork":false,"pushed_at":"2020-12-21T23:26:47.000Z","size":2135,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-04T07:14:15.803Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/codemation.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-11-14T11:30:55.000Z","updated_at":"2022-06-25T04:40:37.000Z","dependencies_parsed_at":"2022-09-14T21:01:49.751Z","dependency_job_id":null,"html_url":"https://github.com/codemation/pyql-cluster","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/codemation/pyql-cluster","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codemation%2Fpyql-cluster","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codemation%2Fpyql-cluster/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codemation%2Fpyql-cluster/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codemation%2Fpyql-cluster/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/codemation","download_url":"https://codeload.github.com/codemation/pyql-cluster/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/codemation%2Fpyql-cluster/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262981613,"owners_count":23394560,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-11T18:59:57.174Z","updated_at":"2025-07-01T14:36:39.404Z","avatar_url":"https://github.com/codemation.png","language":"Python","readme":"# pyql-cluster\nA distributed and scale-in/out REST API application, built with python, which provides users of any language the ability to access, store, and update data, via the Rest PYQL JSON syntax, with a cluster of managed \u0026 distributed table replicas consisting of multiple database types \u0026 locations.\n\n## Key Features\n- Scale-in / out orchestration plane  - As load increases / decreases, endpoints can be scaled up / down. \n- Masterless Scale in / out data table clusters - As load increases, pyql-cluster can add replicas to clusters of tables or decrease on reduced load. Providing read load-balancing, table redundancy, self-healing \n- Data access via REST - access to table data using PYQL JSON query syntax \n- Easy Data sharding via table clusters - data clusters proivde any easy method of creating data table shards with individual scaling capabilities\n- Secure/Shareable - Cluster access secured using tokens for Cluster Owners \u0026 Sharable for other users \n- No downtime Upgrades - Min 3 node cluster \n\n## Quick Start - Try it out\nrequirements: a bootstrapped kubernetes cluster\n\n### Bootstrap pyql-cluster \n\n        $ git clone https://github.com/codemation/pyql-cluster.git\n        $ cd pyql-cluster/k8s/\n        $ kubectl apply -f init_config/\n        configmap/pyql-cluster-config configured\n        persistentvolume/pyql-pv-volume-00 created\n        persistentvolume/pyql-pv-volume-01 created\n        persistentvolume/pyql-pv-volume-02 created\n        secret/pyql-cluster-init-auth created \n\n        # Default secret password is 'abcd1234'\n\n        $ kubectl apply -f init/\n        statefulset.apps/pyql-cluster created\n        service/pyql-cluster-lb created\n\n        $ kubectl get service pyql-cluster-lb\n        NAME              TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE\n        pyql-cluster-lb   NodePort   10.97.143.170   \u003cnone\u003e        80:32543/TCP   4d13h\n\n### Create a new data user: \n\n        $ curl -X POST -H \"Authentication: Basic $(echo 'admin:abcd1234' | base64)\" \\\n            -H \"Content-Type: application/json\" --data '{\"username\":\"aNewUser\",\"email\":\"newUser@aCompany.com\",\"password\":\"abcd1234\"}' \\\n            http://pyql-cluster:32543/auth/user/register\n        {\"message\":\"user created successfully\"}\n\n### Create a join token for user\n\n        $ curl -H \"Authentication: Basic $(echo 'aNewUser:abcd1234' | base64)\" \\\n            http://pyql-cluster:32543/auth/token/join\n        {\"join\":\"eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpZCI6ImU3MjRjODQyLWIzNDQtMTFlYS05NjgxLTYyOGNhNTQ1MTNjYyIsImV4cGlyYXRpb24iOiJqb2luIiwiY3JlYXRlVGltZSI6MTU5MjY5MjE0OC4zNjk1NTE0fQ.NtVVHjfTTuAZXWUqb-RVv2kxAWBpSHZlKUA8Smw1OSk\"}\n\n### Deploy a Data Cluster\n\n\n\n#### Lifecycle of a data-cluster - IMPORTANT - Read before use!\nA data cluster is a cluster of [pyql-rest](https://github.com/codemation/pyql-rest) endpoint tables. This may be all tables, a few or even just one table from a database. \n\nConsiderations:\n* When an endpoint joins a cluster, the joining endpoint decides which tables to add to the cluster. If the cluster already contained other tables, these tables will then be added on the new endpoint and the new table will be added on other existing endpoints. \n* Tables already existing in the cluster will take priority over 'added' tables, meaning if a new endpoint trys to add table 'employees' and the able already existed, the newly added endpoints table 'employees' will be dropped \u0026 synced with the existing tables. This behavior underscores the importance of knowing what tables already exist within a cluster \u0026 configuring separate clusters where conflicts need to be avoided. \n* TO AVOID DATA-LOSS - DO NOT join a cluster with a loaded table if table name already exists in the cluster with different or un-related data. If re-joining, this will cause a resync(only changes will be applied to joining table), if joining for first time, a drop is issued against the table.   \n* Cluster names are unique only with the creating user, so the same cluster name can be used by multiple users\n* Clusters may contain tables from multiple database types (MySql \u0026 Sqlite3 currently supported by pyql-rest endpoints)\n* Endpoints may exist in many different locations and in many different forms, and must be accessible by all pyql-cluster endpoints services requests.\n\n### What type of endpoints can join a data cluster?\npyql-cluster does not manage database credentials, but instead manages pyql-rest endpoint tokens for auth to data-cluster tables in pyql-rest endpoints. This means a joining endpoint must be a [pyql-rest](https://github.com/codemation/pyql-rest) endpoint, the method a pyql-rest endpoint joins a data-cluster can vary:\n* Endpoints of type PYQL_TYPE=K8S expect to join a pyql-cluster immedietly and are configured with parameters to connect to a PYQL_CLUSTER. This method ensure if the pyql-rest endpoint instance is restarted, that it will re-join automatically(important if network path changes). This is the method that k8s pyql-rest replicaStatefulSets use to ensure replica pods always update their path if a pod is terminated and restarted (due to scale down / scale up, rolling upgrade, manual deletion or liveness probe)\n* Endpoints of type PYQL_TYPE=STANDALONE may join a cluster, but must manually invoke the joinCluster API to the pyql-cluster.\n\n### Initializing a Data Cluster\nA data cluster is created using the /cluster/{clusterName}/create API or by invoking /cluster/{clusterName}/join, with /cluster/{clusterName}/join, if the cluster does not yet exist for the user, the cluster is created. See other consideratoins noted above. \n\nA new Data-Cluster (without endpoints / tables) can be used after atleast 1 endpoint is added / created. Currently pyql-cluster does not orchestrate the deployment of new pyql-rest endpoints, but provides templates \u0026 generator scripts for getting started. \n\n#### Deploy data-cluster on K8S or Docker\nData-Cluster endpoints may be deployed as standalone containers or within a kubernetes statefulSet deployment. The power of kubernetes ability to scale in / out, schedule / reschedule failed instances makes this an attractive choice, and is what pyql-cluster \u0026 pyql-rest endpoints are designed primarily to utilize(but not required).\n\n#### Data-Cluster - K8S\nWhen a Data-Cluster is first created, typically this is created by a pyql-rest endpoint which links with an existing database. This is known as a 'seed' deployment, as this may be the original source of some tables in the cluster. These instances are typically deployed with a pre-existing database instance, but can also be deployed as a side-car with a new database instance.    \n\nPre-Requistites:\n1. Registered User in pyql-cluster\n2. JoinToken generated by registered user\n\n[pyql-rest/k8s](https://github.com/codemation/pyql-rest/blob/master/k8s/) provides a generator script for creating \u0026 applying the appropiate configuration files within the current k8s namespace needed for both creating a seeding \u0026 replica deployment. This is demonstated beow:\n\nPull configuration Generator\n\n        $ wget https://github.com/codemation/pyql-rest/blob/master/k8s/generate-pyql-rest.py\n\nGenerator Syntax:\n     \n        # Order of arguments is not important\n        python generate-pyql-rest.py \\ \n            --clusterid \u003cname\u003e # A unique name that should be associated with all deployments (replica \u0026 seed) for the same Data-Cluster, this name does not affect application, but affects k8s config, secret, \u0026 statefulSet file \u0026 object naming. \n            --tables \u003cALL|table1,table2 ..\u003e # What tables are included from database added with endpoint\n            --databases \u003cdatabase\u003e # database rest-endpoint will connect to and joined with Data-Cluster \n            --dbtype \u003cmysql|sqlite\u003e \n            --dbuser \u003cuser\u003e \n            --dbpassword \u003cpw\u003e \n            --dbport \u003cport\u003e # Port dbhost is listening on for db connections\n            --dbhost 83.127.235.150 # host addres pyql-cluster \u0026 pyql-rest instances in K8s will use to access \n            --token \u003ctoken\u003e # Generated join token\n            --clustername \u003ccluster\u003e # name of Data-Cluster to init/join\n            --tag \u003ctag\u003e  # unique special identifier to describe database (deparment, site, location, etc ..). There can be many deployments with the same clusterid, but should only be 1 deployement with this tag\n            --port \u003cpyql_port\u003e # Port which pyql-rest endpoint in k8s will listen for requests\n            --pyqlclustersvc \u003cpyql-cluster service|host:port # example pyql-cluster-lb.default.svc.cluster.local # if deployed in same k8s cluster as pyql-cluster, service name of pyql-cluster should be used, otherwise use pyql-cluster-hostname:port\n\nGenerator Example:\n\n        # assumes an activated python3 env\n\n        $ python generate-pyql-rest.py --clusterid cluster-data1 --tables ALL --databases joshdb --dbtype mysql --dbuser josh --dbpassword abcd1234 --dbport 3306 --dbhost 83.127.235.150 --token eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpZCI6ImViMzJkNDAyLWI2NjgtMTFlYS04ZThjLWVlZGQ2OTIyMGQ4NCIsImV4cGlyYXRpb24iOiJqb2luIiwiY3JlYXRlVGltZSI6MTU5MzAzNzI1OC4yMjY4OTkxfQ.g3sBiIXkEzfurwdGxqPV95hq90I6CO_b_Bhd0TnymAE --clustername data1 --tag sitea --port 80 --pyqlclustersvc pyql-cluster-lb.default.svc.cluster.local\n        configmap/pyql-rest-cluster-config-cluster-data1 created\n        configmap/pyql-rest-seed-db-config-cluster-data1-sitea created\n        persistentvolume/pyql-rest-seed-pv-cluster-data1-sitea created\n        secret/pyql-rest-seed-db-secrets-cluster-data1-sitea created\n        secret/pyql-rest-replica-mysql-secrets-cluster-data1 created\n        configmap/pyql-rest-replica-mysql-config-cluster-data1 created\n        configmap/pyql-rest-replica-config-cluster-data1 created\n        secret/pyql-rest-replica-secrets-cluster-data1 created\n\n        # configuratoin files created\n        $ ls *.yaml\n        pyql-rest-cluster-config-cluster-data1-sitea.yaml        pyql-rest-replica-config-cluster-data1.yaml        pyql-rest-seed-db-config-cluster-data1-sitea.yaml\n        pyql-rest-cluster-seed-ss-init-cluster-data1-sitea.yaml  pyql-rest-replica-mysql-config-cluster-data1.yaml  pyql-rest-seed-pv-cluster-data1-sitea.yaml\n        pyql-rest-cluster-seed-ss-join-cluster-data1-sitea.yaml  pyql-rest-replica-ss-cluster-data1.yaml\n\nGenerator applys configmaps \u0026 secrets right away and writes configmaps to files. Secrets are not written to files to avoid accidental commit into source code. \n\nTo update the existing configmaps, secrets, or statefulSet, either re-run the generator( will overwrite existing files in path), or edit the .yaml config files. \n\n\nDeploy seed \n\n        #current pods\n        $ kubectl get pods\n        NAME             READY   STATUS    RESTARTS   AGE\n        pyql-cluster-0   1/1     Running   0          2m4s\n        pyql-cluster-1   1/1     Running   0          7m4s\n        pyql-cluster-2   1/1     Running   0          4m21s\n\n        # Deploy seed statefulSet\n        $ kubectl apply -f pyql-rest-cluster-seed-ss-init-cluster-data1-sitea.yaml\n        statefulset.apps/pyql-rest-seed-ss-cluster-data1-sitea configured\n\n        # check seed status\n        $ kubectl get pods --selector pyql-rest-cluster-name=data1\n        pyql-rest-seed-ss-cluster-data1-sitea-0   1/1     Running   0          7s\n\nWith the seed deployed, the seed with be accessible via the pyql-cluster URI: http://pyql-cluster/{cluster name}/data1/tables/{table name} APIs for querying \u0026 updating. The data querying APIs are very closely match [pyql-rest](https://github.com/codemation/pyql-rest) in that data is accessed via /cluster/{cluster name}/table/{table name} instead of /db/{db name}/table/{table name}. \n\nDeploy replicas\n\n        # deploy replicas for cluster\n        $ kubectl apply -f pyql-rest-replica-ss-cluster-data1.yaml\n\n        # check pods in cluster\n        $ kubectl get pods --selector pyql-rest-cluster-name=data1\n        pyql-rest-replica-ss-cluster-data1-0      0/2     Pending   0          68s\n        pyql-rest-seed-ss-cluster-data1-sitea-0   1/1     Running   0          2m20s\n\nIf the pod is 'described', the pending status is due to no available persistent volumes, as these must be manually provisioned with a storage class 'pyql-rest-pv-manual-sc' (default, but editable before deployment). The following helper script is also available for provisioning PV's. \n\n        # pull pv provisioner helper script\n        $ wget https://github.com/codemation/pyql-rest/blob/master/k8s/generate_pvs.py\n\n        # usage\n        $ python generate_pvs.py \u003cdesired count\u003e\n        \n        # example\n        $ python generate_pvs.py 4\n        persistentvolume/pyql-rest-pv-00 created\n        persistentvolume/pyql-rest-pv-01 created\n        persistentvolume/pyql-rest-pv-02 created\n        persistentvolume/pyql-rest-pv-03 created\n\n        # to add more, simply increase number\n        $ python generate_pvs.py 6\n        persistentvolume/pyql-rest-pv-00 unchanged\n        persistentvolume/pyql-rest-pv-01 unchanged\n        persistentvolume/pyql-rest-pv-02 unchanged\n        persistentvolume/pyql-rest-pv-03 unchanged\n        persistentvolume/pyql-rest-pv-04 created\n        persistentvolume/pyql-rest-pv-05 created\n    \nReplica Pods should already begin creation matching statefulSet replicas count\n\n        $ kubectl get pods --selector pyql-rest-cluster-name=data1\n        NAME                                      READY   STATUS              RESTARTS   AGE\n        pyql-rest-replica-ss-cluster-data1-0      0/2     ContainerCreating   0          72s\n        pyql-rest-seed-ss-cluster-data1-sitea-0   1/1     Running             0          2m24s\n\n        $ kubectl get pods --selector pyql-rest-cluster-name=data1\n        NAME                                      READY   STATUS    RESTARTS   AGE\n        pyql-rest-replica-ss-cluster-data1-0      2/2     Running   0          2m4s # replica 1\n        pyql-rest-replica-ss-cluster-data1-1      2/2     Running   0          49s  # replica 2 \n        pyql-rest-seed-ss-cluster-data1-sitea-0   1/1     Running   0          3m16s\n\nNote the 2/2 in READY column for the replicas. This is because the pod is supporting 2 containers,  pyql-rest \u0026 mysql as a sidecar container. \n\nThe final step in this deployment is to convert the seed deployment from 'init' to 'join'. This step ensures that the seed endpoint will re-connect to the cluster, when pod is re-scheduled,  if deleted, terminated. \n\n        # convert seed from init -\u003e join\n        $ kubectl apply -f pyql-rest-cluster-seed-ss-join-cluster-data1-sitea.yaml\n        statefulset.apps/pyql-rest-seed-ss-cluster-data1-sitea configured\n\n        # check pods in cluster\n        $ kubectl get pods --selector pyql-rest-cluster-name=data1\n        NAME                                      READY   STATUS        RESTARTS   AGE\n        pyql-rest-replica-ss-cluster-data1-0      2/2     Running       0          25h\n        pyql-rest-replica-ss-cluster-data1-1      2/2     Running       0          25h\n        pyql-rest-seed-ss-cluster-data1-sitea-0   0/1     Terminating   0          25h\n\n        $ kubectl get pods --selector pyql-rest-cluster-name=data1\n        NAME                                      READY   STATUS    RESTARTS   AGE\n        pyql-cluster-0                            1/1     Running   0          2d\n        pyql-cluster-1                            1/1     Running   0          2d\n        pyql-cluster-2                            1/1     Running   0          2d\n        pyql-rest-replica-ss-cluster-data1-0      2/2     Running   0          25h\n        pyql-rest-replica-ss-cluster-data1-1      2/2     Running   0          25h\n        pyql-rest-seed-ss-cluster-data1-sitea-0   1/1     Running   0          5s\n\nDuring termination, access to the data was still acccessible via the replicas, likewise changes could be made and the seed instance will be synced upon re-join to cluster.\n\n#### Docker\n\nThis is a single running instance, which must be managed manually restarted via other tools\n\nRequirements:\n- New or existing Database\n- Volume path specified on container creation, path location must be managed \u0026 re-used manually. \n\nSee [pyql-rest](https://github.com/codemation/pyql-rest) example\n\n\n## Lifecycle of a pyql-cluster\nA pyql-cluster shares similar traits with that of a data-cluster:\n- Comprised of a self-healing, scalable table cluster\n- Masterless \u0026 load-balanced access to cluster tables\n- Generally contrained to a single k8s cluster to benifit from the native service load-balancing.\n\n### Deploy\nA pyql-cluster can be deployed as a single node and scaled to many nodes. Once deployed the pyql-cluster can be scaled out for re-dundancy immedietly or at a later time. \n\n#### 1. Download, edit, and apply init_config \n\n    PYQL_CLUSTER_SVC: Service Path used by pyql-cluster endpoints (k8s pods) to join pyql-cluster. The service name should match with the name defined in[pyql-cluster-statefulSetInit.yaml](./init/pyql-cluster-statefulSetInit.yaml)\n     \n        Syntax  -  pyql-cluster-lb.\u003ck8s-namespace\u003e.svc.\u003ck8scluster.domain\u003e\n        Example - 'pyql-cluster-lb.default.svc.cluster.local'\n\n    PYQL_PORT - TCP Port used to access PYQL_CLUSTER_SVC\n\n        kubectl apply -f 0-pyql-cluster-config.yaml\n        \n        From Source Code:default name-space \u0026 cluster.local\n        kubectl apply -f https://github.com/codemation/pyql-cluster/blob/master/k8s/init_conig/0-pyql-cluster-config.yaml\n\n#### 2. Deploy Persistent volumes which will store pyql-cluster data\n\n    Basic:\n\n        hostPath persistent volume\n        -- \n        apiVersion: v1\n        kind: PersistentVolume\n        metadata:\n        name: pyql-pv-volume-00\n        labels:\n            type: local\n        spec:\n        storageClassName: pyql-pv-manual-sc\n        capacity:\n            storage: 20Gi\n        accessModes:\n            - ReadWriteOnce\n        hostPath:\n            path: \"/mnt/pyql-pv-volume-00\"\n        \n        OR \n\n        #Deploys 3 type hostPath pyql-cluster volumes with storageClassName: pyql-pv-manual-sc\n        kubectl apply -f https://github.com/codemation/pyql-cluster/blob/master/k8s/init_conig/1-pyql-cluster-pv.yaml\n\n        OR\n\n        $ wget https://github.com/codemation/pyql-cluster/blob/master/k8s/init_config/generate_pvs.py\n        $ python3 generate_pvs.py 1\n        persistentvolume/pyql-pv-volume-00 configured\n\n#### 3. Create Init Admin secret\n\n        $ wget https://github.com/codemation/pyql-cluster/blob/master/k8s/init_conig/create_admin_secret.sh\n\n        $ ./create_admin_secret.sh aVerySecretAdminPassword\n        creating secret with encoded password YVZlcnlTZWNyZXRBZG1pblBhc3N3b3Jk\n        secret/pyql-cluster-init-auth configured\n\n#### 4. Deploy Cluster \n\n        #Pull edit \u0026 deploy \n        $ wget https://github.com/codemation/pyql-cluster/blob/master/k8s/init/pyql-cluster-statefulSetInit.yaml\n        $ kubectl apply -f pyql-cluster-statefulSetInit.yaml\n        statefulset.apps/pyql-cluster created\n        service/pyql-cluster-lb created\n\n\n        Or\n\n        # deploy from source\n        $ kubectl apply -f https://github.com/codemation/pyql-cluster/blob/master/k8s/init/pyql-cluster-statefulSetInit.yaml\n        statefulset.apps/pyql-cluster created\n        service/pyql-cluster-lb created\n        \nNote: In this example 'service/pyql-cluster-lb' is created as a type NodePort service which makes the pyql-cluster accessible on the NodePort of each k8s node, but can be modified to use a [LoadBalancer](https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/)\n\n        $ kubectl get service pyql-cluster-lb\n        NAME              TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE\n        pyql-cluster-lb   NodePort   10.97.143.170   \u003cnone\u003e        80:32543/TCP   4d3h\n\n#### 5. Expand Pyql-Cluster (Optional): \nPrevent single pod of failure by adding pyql-cluster endpoints\n\n\nUsing Admin Password - Pull Join Token from http://pyql-cluster-lb/auth/token/join \n\n        $ curl -H 'Content-Type: application/json' -H \"Authentication: Basic $(echo 'admin:aVerySecretAdminPassword' | base64)\" http://pyql-cluster:32543/auth/token/join\n        {\"join\":\"eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpZCI6IjU4Yjk0MTc0LWIyZWUtMTFlYS05N2MzLTk2MDY1YTNhMWE1NSIsImV4cGlyYXRpb24iOiJqb2luIiwiY3JlYXRlVGltZSI6MTU5MjY1NTU4MC4xNzMxOTUxfQ.Z-KytDTrS_1Q4hyScEjgxA7eT5lCJHV3ukhuI2_iYbM\"}\n\n\nUsing Join Token - Create Join Secret\n\n        # Pull token scret creation script\n        $ https://github.com/codemation/pyql-cluster/blob/master/k8s/join_config/create_secret_join_token.sh\n\n        # Generate pyql-cluster-join-token secret\n        $ ./create_secret_join_token.sh eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpZCI6IjU4Yjk0MTc0LWIyZWUtMTFlYS05N2MzLTk2MDY1YTNhMWE1NSIsImV4cGlyYXRpb24iOiJqb2luIiwiY3JlYXRlVGltZSI6MTU5MjY1NTU4MC4xNzMxOTUxfQ.Z-KytDTrS_1Q4hyScEjgxA7eT5lCJHV3ukhuI2_iYbM\n        creating secret with token eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpZCI6IjU4Yjk0MTc0LWIyZWUtMTFlYS05N2MzLTk2MDY1YTNhMWE1NSIsImV4cGlyYXRpb24iOiJqb2luIiwiY3JlYXRlVGltZSI6MTU5MjY1NTU4MC4xNzMxOTUxfQ.Z-KytDTrS_1Q4hyScEjgxA7eT5lCJHV3ukhuI2_iYbM\n        secret/pyql-cluster-join-token configured\n\nAdd new PVs to support new PV endpoints\n\n        $ python3 generate_pvs.py 3\n        persistentvolume/pyql-pv-volume-00 unchanged\n        persistentvolume/pyql-pv-volume-01 configured\n        persistentvolume/pyql-pv-volume-02 configured \n\nBegin expansion\n        \n        # Pull edit \u0026 deploy \n        $ wget https://github.com/codemation/pyql-cluster/blob/master/k8s/pyql-cluster-statefulSet.yaml\n\n        # edit desired number of replicas \n        $ kubectl apply -f pyql-cluster-statefulSet.yaml\n        statefulset.apps/pyql-cluster configured\n\n        OR \n\n        # From Source - Expands Cluster to 3 endpoints - using defaults\n        $ kubectl apply -f https://github.com/codemation/pyql-cluster/blob/master/k8s/pyql-cluster-statefulSet.yaml\n\n        $ kubectl apply -f pyql-cluster-statefulSet.yaml \n        statefulset.apps/pyql-cluster configured\n\nMonitor Expansion - each node make take up to 3 minutes to be 'ready' which allows endpoint to service pyql-cluster requests. \n\n        $ kubectl get pods\n        NAME             READY   STATUS    RESTARTS   AGE\n        pyql-cluster-0   1/1     Running   0          31m\n        pyql-cluster-1   0/1     Running   0          103s\n\n        $ kubectl get pods\n        NAME             READY   STATUS    RESTARTS   AGE\n        pyql-cluster-0   1/1     Running   0          2m\n        pyql-cluster-1   1/1     Running   0          5m38s\n        pyql-cluster-2   1/1     Running   0          3m51s\n\n### Upgrades\nUpgrades on pyql-clusters of size 3 or greater, can be performed with zero downtime with the help of rolling upgrades in k8s \u0026 pyql-cluster quorum mechaics. At least 2 endpoints out of every 3 must be online for 'quorum' which permits service requests. \n\nSteps To upgrade / downgrade the version of pyql-cluster:  \n1. Select desired image from [DockerHub](https://hub.docker.com/r/joshjamison/pyql-cluster/tags)\n2. Edit statefulset statefulSet with new image:tag\n3. Apply change\n\n        # PYQL UPGRADE - to selected version in pyql-cluster-statefulSet.yaml\n        $ kubectl apply -f pyql-cluster-statefulSet.yaml\n\n        # PYQL UPGRADE from source to latest\n        $ kubectl apply -f https://github.com/codemation/pyql-cluster/blob/master/k8s/pyql-cluster-statefulSet.yaml\n\n        # Rolling Upgrade Begin \n        $ kubectl get pods\n        NAME             READY   STATUS        RESTARTS   AGE\n        pyql-cluster-0   1/1     Running       0          51m\n        pyql-cluster-1   1/1     Running       0          55m\n        pyql-cluster-2   1/1     Terminating   0          53m\n\n        # After node is online with new version \u0026 ready, next node will begin\n        $ kubectl get pods\n        NAME             READY   STATUS        RESTARTS   AGE\n        pyql-cluster-0   1/1     Running       0          53m\n        pyql-cluster-1   1/1     Terminating   0          56m\n        pyql-cluster-2   1/1     Running       0          72s\n\n        # All nodes report ready once completed\n        $ kubectl get pods\n        NAME             READY   STATUS    RESTARTS   AGE\n        pyql-cluster-0   1/1     Running   0          2m20s\n        pyql-cluster-1   1/1     Running   0          5m8s\n        pyql-cluster-2   1/1     Running   0          9m40s\n\n### Node Outage\npyql-cluster can continue to service data-cluster \u0026 pyql-cluster requests as long as the pyql-cluster maintains a 2/3 quorum with cluster members. A cluster of 3 can support a single endpoint outage(termination / network connectivity, other). A cluster of 6 can support 2 endpoint failures and so on ... \n\n## PYQL-CLUSTER API REFERENCE\nRequest Content-Type \u0026 Response Content-Type will always default to application/json\n\n| ACTION | HTTP Verb | Path             | Auth Required | Request body | Example response body |\n|--------|-----------|------------------|---------------|--------------|-----------------------|\n| Register a new user in pyql-cluster | POST | /auth/{user or admin}/register | pyql | {\"username\":\"aNewUser\",\"email\":\"newUser@aCompany.com\",\"password\":\"abcd1234\"} | {\"message\":\"user created successfully\"} |\n| Create a user authenticatoin token, to be used in place of user/pw in header auth, expires after 3600 seconds | GET | /auth/token/user | cluster | none | {\"token\": \"token....\" |\n| Create a user join token, can only be used to join a cluster owned by user | GET | /auth/token/join | cluster | none | {\"join\": \"token...\"} |\n| Get all rows from cluster table | GET | /cluster/{cluster}/table/{table}' | cluster | none | {\"data\":[ \u003cbr\u003e{\"department_id\":1001,\"id\":100101,\"name\":\"Director\"}, \u003cbr\u003e{\"department_id\":1001,\"id\":100102,\"name\":\"Manager\"}, \u003cbr\u003e{\"department_id\":1001,\"id\":100103,\"name\":\"Rep\"}, \u003cbr\u003e{\"department_id\":1001,\"id\":100104,\"name\":\"Intern\"},  \u003cbr\u003e.. ]} |\n| Create row in cluster table | POST, PUT | /cluster/{cluster}/table/{table}' | cluster | {\"afterHours\":true,\"date\":\"2006-01-08\",\"price\":33.16,\"qty\":null,\"symbol\":null,\"trans\":{\"condition\":{\"limit\":\"34.00\",\"time\":\"EndOfTradingDay\"},\"type\":\"BUY\"}}  | {\u003cbr\u003e \"consistency\":true, \u003cbr\u003e \"message\":{\u003cbr\u003e \"153a4f96-b9d9-11ea-93ec-feed8b92afdf\":{\"content\":{\"message\":{\"message\":\"items added\",\"status\":200},\"status\":200}\u003cbr\u003e.... \u003cbr\u003e} |\n| Get all rows from cluster table | GET | /cluster/{cluster}/table/{table}/select' | cluster | none | {\"data\":[\u003cbr\u003e {\"id\":1000,\"name\":\"Clara Franklin\",\"position_id\":100101},\u003cbr\u003e{\"id\":1001,\"name\":\"Eli Carson\",\"position_id\":100102},\u003cbr\u003e{\"id\":1002,\"name\":\"Joe Smith\",\"position_id\":100102}, \u003cbr\u003e ..]} |\n| Get Specfied Rows from cluster table \u003cbr\u003e\u003cbr\u003eOptional: matching specific column values \u003cbr\u003e\u003cbr\u003eOptional: Join Specified tables  | POST | /cluster/{cluster}/table/{table}/select' | cluster |  ## Example 1\u003cbr\u003e {\"select\":[\"*\"],\"where\":{\"column1\":\"value1\"}} \u003cbr\u003e\u003cbr\u003e ## Example2 \u003cbr\u003e{\"select\":[\"*\"],\"join\": \"positions\"} \u003cbr\u003e Other examples at [pyql-rest](https://github.com/codemation/pyql-rest)  | ## Example 1 \u003cbr\u003e  {\"data\":[\u003cbr\u003e{\"id\":1000,\"name\":\"Clara Franklin\",\"position_id\":100101}, \u003cbr\u003e{\"id\":1001,\"name\":\"Eli Carson\",\"position_id\":100102},\u003cbr\u003e{\"id\":1002,\"name\":\"Joe Smith\",\"position_id\":100102}, \u003cbr\u003e ..]} \u003cbr\u003e\u003cbr\u003e ## Example 2 \u003cbr\u003e{\"data\":[\u003cbr\u003e{\"employees.id\":1000,\"employees.name\":\"Clara Franklin\",\"employees.position_id\":100101,\"positions.department_id\":1001,\"positions.name\":\"Director\"}, \u003cbr\u003e{\"employees.id\":1001,\"employees.name\":\"Eli Carson\",\"employees.position_id\":100102,\"positions.department_id\":1001,\"positions.name\":\"Manager\"},\u003cbr\u003e .. ]} |\n| Get Row with Primary Key Value | GET | /cluster/{cluster}/table/{table}/{key} | cluster | none | {\"data\":[{\"id\":1000,\"name\":\"Clara Franklin\",\"position_id\":100101}]} |\n| Update Row with Primary Key Value | POST | /cluster/{cluster}/table/{table}/{key}  | cluster | {\"name\": \"Clara Franklin-Rogers\"} | {\u003cbr\u003e\"consistency\":true,\u003cbr\u003e\"message\":{\u003cbr\u003e\"153a4f96-b9d9-11ea-93ec-feed8b92afdf\":{\"content\":{\"message\":{\"message\":\"OK\",\"status\":200},\"status\":200},\"status\":200}, .. \u003cbr\u003e... \u003cbr\u003e} |\n| Delete Row with Primary Key Value | DELETE | /cluster/{cluster}/table/{table}/{key}  | cluster | none | {\"consistency\":true,\"message\":{\"153a4f96-b9d9-11ea-93ec-feed8b92afdf\":{\"content\":{\"message\":{\"message\":\"OK\",\"status\":200},\"status\":200},\"status\":200},\u003cbr\u003e... \u003cbr\u003e}|\n| Create row in cluster table | POST | /cluster/{cluster}/table/{table}/insert | cluster | {\"afterHours\":true,\"date\":\"2006-01-09\",\"price\":32.16,\"qty\":null,\"symbol\":null,\"trans\":{\"condition\":{\"limit\":\"34.00\",\"time\":\"EndOfTradingDay\"},\"type\":\"BUY\"}}  | {\u003cbr\u003e \"consistency\":true, \u003cbr\u003e \"message\":{\u003cbr\u003e \"153a4f96-b9d9-11ea-93ec-feed8b92afdf\":{\"content\":{\"message\":{\"message\":\"items added\",\"status\":200},\"status\":200}\u003cbr\u003e.... \u003cbr\u003e} |\n| Update row in cluster table matching condition | POST | /cluster/{cluster}/table/{table}/update | cluster | {\"set\":{\"position_id\":200102},\"where\":{\"id\":1001}} | {\u003cbr\u003e\"consistency\":true,\u003cbr\u003e\"message\":{\u003cbr\u003e\"153a4f96-b9d9-11ea-93ec-feed8b92afdf\":{\"content\":{\"message\":{\"message\":\"OK\",\"status\":200},\"status\":200},\"status\":200}, .. \u003cbr\u003e... \u003cbr\u003e}}  |\n| Delete row in cluster table  matching condition | POST | /cluster/{cluster}/table/{table}/delete | cluster | {\"where\":{\"id\": 1003}} | {\"consistency\":true,\"message\":{\"153a4f96-b9d9-11ea-93ec-feed8b92afdf\":{\"content\":{\"message\":{\"message\":\"OK\",\"status\":200},\"status\":200},\"status\":200},\u003cbr\u003e... \u003cbr\u003e} |\n| Get cluster table config | GET | /cluster/{cluster}/table/{table}/config | cluster | none | {\"employees\":{\"columns\":[{\"mods\":\"NOT NULL\",\"name\":\"id\",\"type\":\"int\"},{\"mods\":\"\",\"name\":\"name\",\"type\":\"str\"},{\"mods\":\"DEFAULT NULL\",\"name\":\"position_id\",\"type\":\"int\"}],\"foreign_keys\":{\"position_id\":{\"mods\":\" ON DELETE CASCADE ON UPDATE CASCADE\",\"ref\":\"id\",\"table\":\"positions\"}},\"primary_key\":\"id\"}}|\n| Get all cluster tables config | GET | /cluster/{cluster}/tables | cluster | none | {\"departments\":{\"columns\":[{\"mods\":\"NOT NULL\",\"name\":\"id\",\"type\":\"int\"},{\"mods\":\"\",\"name\":\"name\",\"type\":\"str\"}],\"foreign_keys\":null,\"primary_key\":\"id\"},\"employees\":{\"columns\":[{\"mods\":\"NOT NULL\",\"name\":\"id\",\"type\":\"int\"},{\"mods\":\"\",\"name\":\"name\",\"type\":\"str\"},{\"mods\":\"DEFAULT NULL\",\"name\":\"position_id\",\"type\":\"int\"}],\"foreign_keys\":{\"position_id\":{\"mods\":\" ON DELETE CASCADE ON UPDATE CASCADE\",\"ref\":\"id\",\"table\":\"positions\"}},\"primary_key\":\"id\"}, \u003cbr\u003e...\u003cbr\u003e} |\n| Join Data Cluster | POST | /cluster/{cluster name}\u003e/join | cluster join_token  | {\u003cbr\u003e \"name\": \"192-168-231-226\",\u003cbr\u003e  \"path\": \"192.168.231.226:80\",\u003cbr\u003e    \"token\":\"token.....\",\u003cbr\u003e \"database\": {\"name\": \"company\", \"uuid\": \"bf4a5a9a-ba00-11ea-97d8-5647a99b32bb\"},\u003cbr\u003e \"tables\": [],\u003cbr\u003e \"consistency\": [\"employees\", \"positions\", \"departments\"]\u003cbr\u003e} | {\"message\": \"join cluster {cluster name} for endpoint 192-168-231-226 completed successfully\")} |\n\nFor advanced pyql json queries, join, multi-join syntax, and more pyql json examples, see [pyql-rest](https://github.com/codemation/pyql-rest). Syntax for pyql json queries does not differ between pyql-cluster and [pyql-rest](https://github.com/codemation/pyql-rest), only the API URI /cluster/{cluster}/table/{table} versus /db/{database}/table/{table} \n\n## PYQL-CLUSTER INTERNAL API REFERENCE\nRequest Content-Type \u0026 Response Content-Type will always default to application/json\n\n| ACTION | HTTP Verb | Path             | Auth Required | Request body | Example response body |\n|--------|-----------|------------------|---------------|--------------|-----------------------|\n| Updates a local pyql endpoint with a new local encryption/decryption key, used by key rotation or initial config | POST | /auth/key/{cluster or local} | local | {'PYQL_LOCAL_TOKEN_KEY': 'key....'} or {'PYQL_CLUSTER_TOKEN_KEY': 'key....'} | {\"message\": \"{location} updated successfully with {key}\")} |\n|  Used primary to update joining nodes with a PYQL_CLUSTER_SERVICE_TOKEN so joining node can pull and set its PYQL_CLUSTER_TOKEN_KEY | POST | /auth/setup/cluster | local | {\"PYQL_CLUSTER_SERVICE_TOKEN\": \"token....\"} | {\"message\": \"{location} updated successfully with {key}\")} |\n| Get current PYQL_CLUSTER_SERVICE_TOKEN or PYQL_CLUSTER_SERVICE_TOKEN | GET | /auth/token/{cluster or local} | pyql | none | {\"PYQL_{local or cluster}_SERVICE_TOKEN\": \"token....} |\n| Get current PYQL_CLUSTER_TOKEN_KEY or PYQL_CLUSTER_TOKEN_KEY | GET | /auth/key/{cluster or local} | pyql | none | {\"PYQL_{local or cluster}_TOKEN_KEY\": \"key....} |\n| GET endpoint UUID | GET | /pyql/node | none | none | {\"uuid\": \"node_uuid...\" |\n| Check the readieness of a pyql endpoint by re-triggering a quorum check \u0026 verifying local state table is in_sync | GET | /cluster/pyql/ready | none | none | {\"health\":\"healthy\",\"inQuorum\":true,\"lastUpdateTime\":1593385795.2684991,\"missing\":{\"nodes\":[]},\"node\":\"c49d32b4-b66e-11ea-b31d-eef4a54d64d8\",\"nodes\":{\"nodes\":[\"276eeb80-b66f-11ea-81a6-d678bdb9f7b8\",\"779d5b3c-b66f-11ea-aebe-4ecfe07dbf34\",\"c49d32b4-b66e-11ea-b31d-eef4a54d64d8\"]},\"ready\":true} |\n| Triggers a quorum check on each node in pyql-cluster | POST | /pyql/quorum/check | pyql | none | {\"message\":\"cluster_quorum_check completed on c49d32b4-b66e-11ea-b31d-eef4a54d64d8\",\"results\":{\"276eeb80-b66f-11ea-81a6-d678bdb9f7b8\":{\"content\":{\"message\":\"cluster_quorum_update on node 276eeb80-b66f-11ea-81a6-d678bdb9f7b8 updated successfully\",\"quorum\":{\"health\":\"healthy\",\"inQuorum\":true,\"lastUpdateTime\":1593386168.311425,\"missing\":{\"nodes\":[]},\"node\":\"276eeb80-b66f-11ea-81a6-d678bdb9f7b8\",\"nodes\":{\"nodes\":[\"c49d32b4-b66e-11ea-b31d-eef4a54d64d8\",\"779d5b3c-b66f-11ea-aebe-4ecfe07dbf34\",\"276eeb80-b66f-11ea-81a6-d678bdb9f7b8\"]},\"ready\":true}},\"status\":200}, ...} |\n| Get current quorum status of endpoint | GET | /pyql/quorum | local | none | {\"health\":\"healthy\",\"inQuorum\":true,\"lastUpdateTime\":1593386168.311425,\"missing\":{\"nodes\":[]},\"node\":\"276eeb80-b66f-11ea-81a6-d678bdb9f7b8\",\"nodes\":{\"nodes\":[\"c49d32b4-b66e-11ea-b31d-eef4a54d64d8\",\"779d5b3c-b66f-11ea-aebe-4ecfe07dbf34\",\"276eeb80-b66f-11ea-81a6-d678bdb9f7b8\"]} |\n| Update quorum status of endpoint | POST  | /pyql/quorum | local | none | {\"health\":\"healthy\",\"inQuorum\":true,\"lastUpdateTime\":1593386168.311425,\"missing\":{\"nodes\":[]},\"node\":\"276eeb80-b66f-11ea-81a6-d678bdb9f7b8\",\"nodes\":{\"nodes\":[\"c49d32b4-b66e-11ea-b31d-eef4a54d64d8\",\"779d5b3c-b66f-11ea-aebe-4ecfe07dbf34\",\"276eeb80-b66f-11ea-81a6-d678bdb9f7b8\"]} |\n| Get endpoint paths to cluster tables | GET | /cluster/{cluster_uuid}/table/{table}/path | pyql | body | {\"in_sync\":{\"276eeb80-b66f-11ea-81a6-d678bdb9f7b8\":\"http://192.168.200.225:80/db/cluster/table/state\",\"779d5b3c-b66f-11ea-aebe-4ecfe07dbf34\":\"http://192.168.231.201:80/db/cluster/table/state\",\"c49d32b4-b66e-11ea-b31d-eef4a54d64d8\":\"http://192.168.231.238:80/db/cluster/table/state\"},\"out_of_sync\":{}}|\n| Get information on all cluster table endpoints | GET | /cluster/{cluster_uuid}/table/{table}/endpoints | pyql | none | {\"clusterName\":\"pyql\",\"in_sync\":{\"276eeb80-b66f-11ea-81a6-d678bdb9f7b8\":{\"cluster\":\"8eac4730-b66e-11ea-ac85-6ae89b450a37\",\"dbname\":\"cluster\",\"in_sync\":true,\"lastModTime\":0.0,\"name\":\"276eeb80-b66f-11ea-81a6-d678bdb9f7b8state\",\"path\":\"192.168.200.225:80\",\"state\":\"loaded\",\"table_name\":\"state\",\"token\":\"eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpZCI6IjI3YTAxNDU4LWI2NmYtMTFlYS04MWE2LWQ2NzhiZGI5ZjdiOCIsImV4cGlyYXRpb24iOiJuZXZlciJ9.fCRVSSLiAe6bWE3s9reKd6csGBsEr5h_NZ6NspYEEGQ\",\"uuid\":\"276eeb80-b66f-11ea-81a6-d678bdb9f7b8\"}, .. }},\"out_of_sync\":{}} |\n|  Get information on specific table endpoint | GET | /cluster/{cluster_uuid}/table/{table}/{} | local_pyql_cluster | body | body_response |","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodemation%2Fpyql-cluster","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcodemation%2Fpyql-cluster","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcodemation%2Fpyql-cluster/lists"}