Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/pyar/pyar_infra
Python Argentina Infrastructure
https://github.com/pyar/pyar_infra
Last synced: about 1 month ago
JSON representation
Python Argentina Infrastructure
- Host: GitHub
- URL: https://github.com/pyar/pyar_infra
- Owner: PyAr
- Created: 2018-04-29T13:11:00.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2024-02-08T12:18:59.000Z (11 months ago)
- Last Synced: 2024-02-08T13:33:49.193Z (11 months ago)
- Language: Mustache
- Size: 338 KB
- Stars: 6
- Watchers: 7
- Forks: 1
- Open Issues: 24
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Python Argentina Infrastructure
This is the repository with all the code and documentation to handle PyAr infrastructure
We are working with [kubernetes](http://kubernetes.io/) in Azure [aks](https://docs.microsoft.com/en-us/azure/aks/)
We use [Helm](https://www.helm.sh/) as a package manager.
[Step-by-step guide to deploy de cluster](docs/k8s.md)
## HTTPS config
We are using HTTPS with [Let's Encrypt](https://letsencrypt.org/)
Settings details at: https://docs.microsoft.com/en-us/azure/aks/ingress
## Redirecter.
We have lot of domains. But python.org.ar is our principal.
To handle redirects from other domains we have two models:
1. nginx ingress rules, different services configured with files in the `redirects` directory
- first time:
kubectl create -f redirects/prueba.yaml- after any change:
kubectl apply -f redirects/prueba.yaml
- to see what's there:
kubectl get pods --namespace=ingress-basic
2. nginx server, handling `redirecter.python.org.ar`, the configuration is stored in a config-map: `stable/pyar-rewrites/templates/config_map.yaml`
- to deploy it run:
```bash
helm upgrade --install --wait pyar-rewrites stable/pyar-rewrites
```- check its public IP (`LoadBalancer Ingress` after doing `kubectl describe services pyar-rewrites`) and load it in Cloudflare for the `redirecter` entry.
## The Database, a PostgreSQL cluster
Using https://github.com/helm/charts/tree/master/stable/postgresql
### Deploy:
El siguiente comando hace el deploy. NOTA: NO tiene que estar el secreto `pgcluster-postgresql` al momento de deployar PSQL (se crea en ese proceso).
```bash
helm upgrade --install --wait -f values/production/postgres_cluster.yaml pgcluster oci://registry-1.docker.io/bitnamicharts/postgresql
```This cluster is using a PersistentVolumeClaim and a "lock" is created manually in azure to prevent unintencional deletes. Detail about locks: https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-group-lock-resources
To set the backup:
```bash
kubectl apply -f k8s/pgsql_bkps_jobs/pg-storage-class.yaml
kubectl apply -f k8s/pgsql_bkps_jobs/pg-persistent-volume-claim.yaml
kubectl apply -f k8s/pgsql_bkps_jobs/pg-backup-cronJob.yaml
```### Connect to the cluster
```bash
# get the password
export POSTGRES_PASSWORD=$(kubectl get secret --namespace default pgcluster-postgresql -o jsonpath="{.data.postgres-password}" | base64 --decode)
# connect
kubectl run pgcluster-postgresql-client --rm --tty -i --restart='Never' --namespace default --image docker.io/bitnami/postgresql:11.5.0-debian-9-r84 --env="PGPASSWORD=$POSTGRES_PASSWORD" --command -- psql --host pgcluster-postgresql -U postgres -p 5432
```### Configuration
We have to create the databases and users manually
### Restore backups
1. Download the Backup file from Azure Blob Storage
1. Create a console to the PostgreSQL cluster
```bash
# get the password
export POSTGRES_PASSWORD=$(kubectl get secret --namespace default pgcluster-postgresql -o jsonpath="{.data.postgres-password}" | base64 --decode)
# connect
kubectl run pgcluster-postgresql-client --rm --tty -i --restart='Never' --namespace default --image docker.io/bitnami/postgresql:11.5.0-debian-9-r84 --env="PGPASSWORD=$POSTGRES_PASSWORD" --command -- /bin/bash
```1. On a new local console, copy the local downloaded file to the cluster
```bash
kubectl cp *.dump pgcluster-postgresql-client:/tmp/backup
```1. On the existing console to the PostgreSQL cluster run the restore command. Change the `CHANGE_THE_DATABASE` for the correct value
```bash
I have no name!@pgcluster-postgresql-client:/$ pg_restore --host pgcluster-postgresql -U postgres --d CHANGE_THE_DATABASE /tmp/backup
```## Python Argentina community website
http://www.python.org.ar
```bash
helm upgrade --install --wait --timeout 120s --values values/production/pyarweb.yaml pyarweb-production stable/pyarweb
```## Wiki
Using https://github.com/helm/charts/tree/master/testing/wiki
Staging:
```bash
helm upgrade --install --wait --timeout 120s --values values/staging/pyar-wiki.yaml staging-wiki stable/pyar-wiki --debug
```Production:
```bash
helm upgrade --install --wait --timeout 120s --values values/production/pyar-wiki.yaml prod-wiki stable/pyar-wiki --debug
```## Asociación Civil administration. (asoc_members)
[Production](https://admin.ac.python.org.ar)
```bash
helm upgrade --install --wait --timeout 120s --values values/production/asoc_members.yaml production-admin stable/asoc-members
```## Join Captcha bot
```bash
helm upgrade --install --wait --timeout 120s --values values/production/join_captcha_bot.yaml captcha-bot-production stable/join_captcha_bot
```Once up, talk through Telegram with the bot itself and issue: `/allowgroup add CHAT_ID` (the CHAT_ID can be seen in the logs doing something similar to `kubectl logs captcha-bot-production-5d99c5595d-8wcbb`).
## Events site (EventoL)
https://eventos.python.org.ar
Events site, using [EventoL](https://github.com/eventoL/eventoL). We use it to host PyDays, PyCon, Pycamp and other events.
### Staging
See instructions on how to copy the static files to Azure, but bear in mind that they might collide with production ones.
Final command to deploy to staging:
```bash
helm upgrade --install --wait --timeout 60000 --values values/staging/eventol.yaml staging-eventos stable/eventol
```### Production
1. We need to get the static files from Eventol docker image and push them to azure. First get a docker container with all the statics inside (the version might change, check the values/production/eventol.yaml file):
```
$ docker run --name eventol -it registry.gitlab.com/eventol/eventol/releases:v2.3.4 /bin/ash
```2. In other terminal, bring those static files to your dev machine:
```
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
2e88bd843642 eventol/eventol "/bin/ash" 41 seconds ago Up About a minute 8000/tcp eventol
$ mkdir /tmp/eventolstatic
$ cd /tmp/eventolstatic
$ docker cp CONTAINER_ID:/usr/src/app/eventol/static .
```(at this point you can let go the container from (1) )
3. Upload the static files to Azure Storage
```
cd /tmp/eventolstatic/
az storage copy -s static -d 'https://pyareventol.file.core.windows.net/eventol-prod-static/' --recursive
```3. Deploy to [Production:](https://eventos.python.org.ar)
```bash
helm upgrade --install --wait --timeout 120s --values values/production/eventol.yaml production-eventos stable/eventol
```