{"id":16288423,"url":"https://github.com/wuxxin/infra-shared","last_synced_at":"2025-03-20T03:30:38.845Z","repository":{"id":166037400,"uuid":"641465553","full_name":"wuxxin/infra-shared","owner":"wuxxin","description":"Software Defined Git Operated Infrastructure","archived":false,"fork":false,"pushed_at":"2025-03-03T23:10:25.000Z","size":799,"stargazers_count":5,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-17T14:03:08.612Z","etag":null,"topics":["butane","coreos","fcos","gitops","iaas","mtls","pulumi","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wuxxin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-16T14:22:36.000Z","updated_at":"2025-03-03T23:10:28.000Z","dependencies_parsed_at":null,"dependency_job_id":"128aaa8b-1d05-4cad-8b7d-a2742c82f4c1","html_url":"https://github.com/wuxxin/infra-shared","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wuxxin%2Finfra-shared","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wuxxin%2Finfra-shared/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wuxxin%2Finfra-shared/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wuxxin%2Finfra-shared/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wuxxin","download_url":"https://codeload.github.com/wuxxin/infra-shared/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244543713,"owners_count":20469547,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["butane","coreos","fcos","gitops","iaas","mtls","pulumi","python"],"created_at":"2024-10-10T19:48:16.502Z","updated_at":"2025-03-20T03:30:38.840Z","avatar_url":"https://github.com/wuxxin.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# infra-shared\n\n## Software Defined Git Operated Infrastructure\n\nReusables of a learning project by rewriting parts of my home infrastructure as\n\na **Pulumi** (Terraform-ish) and **Fedora Coreos** based **Gitops** Project in **Python**.\n\n- See [safe](examples/safe) for usage in an example project\n\n### Quick start\n\ncreate a base project, lock and install build requirements,\ninstall and configure a simulation of the targets\n\n```sh\nmkdir -p example; cd example; git init\ngit submodule add https://github.com/wuxxin/infra-shared.git infra\ninfra/scripts/create_skeleton.sh --yes\nmake sim-up\n```\n\n**Congratulations!**\n\nYou have just created two TLS Certificates and an SSH Keypair in a very fancy way!\n\nSee the [examples](examples/) for code of what else can be done with it\n\n### Features\n\n- **Appliance** based on **Fedora-CoreOS Linux** - updating, minimal, monolithic, container-focused operating system\n    - **Setup**: Bootstrap and Reconfiguration of CoreOS with **Jinja templated butane** files\n    - **Reconfiguration**: `update-system-config*`\n        - Fast (~4s) reconfiguration using saltstack and butane to salt translation\n    - **Single Container**: `podman-systemd.unit`\n        - `container*` - run systemd container units using podman-quadlet\n    - **Compose Container**: `compose.yml`\n        - `compose*` - run multi-container applications defined using a compose file\n    - **nSpawn OS-Container**: `systemd-nspawn`\n        - `nspawn*` - run any linux OS in a light-weight system container\n    - **tls/http/web FrontEnd**: `traefik`\n        - using container, compose and nspawn labels for dynamic configuration\n    - **DNS Resolver**: `unbound`\n        - using container for local DNSSEC capable recursive DNS-Resolver\n- **TLS Certificate-Authority**, TLS Certificates and **SSH**-Certificates\n- **SSH** copy/deploy/execute functions, local and remote **Salt-Call**\n- **serve** configuration **HTTPS** payloads, request a port forwarding\n- **write** image to **removable storage** specified by serial_number\n- build Embedded-OS Images and IOT Images\n    - **Raspberry PI Extras** - Eeprom, U-Boot and UEFI bios files\n    - **Openwrt Linux** - Network Device Distribution for Router and other network devices\n\n### Technologies\n\n**Need to know** technologies (to write Deployment and Docs):\n\n- Basic Knowledge of `Python, Yaml, Jinja, Systemd Service, Containerfile, Markdown`\n\n**Advanced functionality** available with knowledge of:\n\n- Pulumi, Butane, more Systemd, Fcos, Saltstack, Podman, compose.yml, makefile, pyproject.toml, libvirt, Bash, Mkdocs, Mermaid, Jupyter or Marimo Notebooks\n\nProvision can be run on **Arch** Linux, Manjaro Linux or as **Container Image**.\n\n#### Tools used\n\n- `pulumi` - imperativ infrastructure delaration using python\n- `fcos` - Fedora-CoreOS, minimal OS with `clevis` (sss,tang,tpm) storage unlock\n- `butane` - create fcos `ignition` configs using `jinja` enhanced butane yaml\n- `systemd` - service, socker, path, timer, nspawn machine container\n- `podman` - build Container and NSpawn images, run Container using quadlet systemd container\n- `saltstack`\n    - local build environments and local services\n    - remote fcos config update using butane to saltstack translation and execution\n- `mkdocs` - documentation using markdown and mermaid\n- `libvirt` - simulation of machines using the virtualization api supporting qemu and kvm\n- `tang` - server used for getting a key shard for unattended encrypted storage unlock on boot\n- `age` - ssh keys based encryption of production files and pulumi master password\n- `uv`- virtualenv management using pyproject.toml and uv.lock\n\n### Usage\n\n#### List available Makefile targets/commands\n\n```sh\nmake\n```\n\n#### Bootstrap skeleton files to a new repo\n\n- from current directory, eg. pwd=~/code\n\n```sh\nproject_name=example\ncurrent_dir=$(pwd)\nproject_dir=${current_dir}/${project_name}\nmkdir -p ${project_dir}\ncd ${project_dir}\ngit init\ngit submodule add https://github.com/wuxxin/infra-shared.git infra\ninfra/create_skeleton.sh --yes\n```\n\n- `create_skeleton.sh` creates default dirs and files in the project_dir\n    - use `cat infra/create_skeleton.sh` to inspect script before running it\n    - directories created:\n        - _docs_, _state_, _target_ with an empty _.gitkeep_ file inside\n    - files created:\n        - README.md, \\_\\_main\\_\\_.py, Pulumi.yaml, Makefile, pyproject.toml\n        - config-template.yaml, .gitignore, mkdocs.yml, empty authorized_keys\n\n#### Install build requirements\n\n- on arch linux or manjaro linux\n\n```sh\nmake install-requirements\n```\n\n- on other linux, use a provision container.\n\nThis needs podman or docker already installed on host.\n\nFor the simulation environment with libvirt the host system must also have a configured libvirt.\n\n```sh\n# Either: build container using `sudo podman build`\nmake provision-client\n\n# Or: build container using any other container tool\n# - replace \"docker\" with the preferred container build call\ncd infra/Containerfile/provision-client \u0026\u0026 \\\n    docker build -t provision-client:latest $(pwd)\n\n# call provision shell(defaults to /usr/bin/bash interactive shell)\n# defaults to podman, but can be overriden with DOCKER_CMD=executable\nDOCKER_CMD=docker infra/scripts/provision_shell.sh\n# use exit to return to base shell\n```\n\n#### Build documentation\n\n```sh\nmake docs\n# build infra-shared documentation\nmake docs-infra\n```\n\n#### Create/build/install simulation target\n\n```sh\nmake sim-up\n```\n\n#### Show/use root and provision cert\n\n```sh\nmake sim-show args=\"ca_factory\" | jq \".root_cert_pem\" -r | \\\n    openssl x509 -in /dev/stdin -noout -text\nmake sim-show args=\"ca_factory\" | jq \".provision_cert_pem\" -r | \\\n    openssl x509 -in /dev/stdin -noout -text\n```\n\n#### Manual pulumi invocation\n\n```sh\nexport PULUMI_SKIP_UPDATE_CHECK=1\nexport PULUMI_CONFIG_PASSPHRASE=sim\n\npulumi stack select sim\npulumi about\n```\n\n#### Execute in provision python environment\n\n```sh\nuv run ipython\n```\n\n#### Sim stack: destroy, cleanup, re/create\n\n```sh\nmake sim-clean\n# in case something happend while destroying sim stack\nmake sim__ args=\"stack rm --force\"; rm Pulumi.sim.yaml\n# recreate stack\nmake sim-create\n```\n\n#### test if changes would compute before applying\n\n```sh\nmake sim-preview\n# if list of changes looks good, apply them\nmake sim-up\n\n```\n\n#### cancel an currently running/stuck pulumi update\n\n```sh\n# \"error: the stack is currently locked by 1 lock(s).\"\n# \"Either wait for the other process(es) to end or delete the lock file with `pulumi cancel`.\"\nmake sim__ args=\"cancel\"\n```\n\n#### show resource output as json\n\n```sh\nmake sim-show\n```\n\n#### show resource output key list as yaml\n\n```sh\nmake sim-list\n```\n\n#### show resource output data as colorized formatted json or yaml\n\n```sh\n# use highlight and less\nmake sim-show | highlight --syntax json -O ansi | less\n# use bat for integrated highlight plus pager\nmake sim-show | bat -l json\n```\n\n### Production\n\n#### Add SSH Keys of GitOps Developer\n\n```sh\n# eg. add the own ssh public key in project_dir/authorized_keys\ncat ~/.ssh/id_rsa.pub \u003e\u003e authorized_keys\n```\n\n#### Create stack\n\n```sh\nmake prod-create\nmake prod__ args=\"preview --suppress-outputs\"\nmake prod__ args=up\n```\n\n### Credits\n\n- Inspired and impressed by [deuill/coreos-home-server](https://github.com/deuill/coreos-home-server)\n\n### License\n\n```text\nAll code in this repository is covered by the terms of the Apache 2.0 License,\nthe full text of which can be found in the LICENSE file.\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwuxxin%2Finfra-shared","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwuxxin%2Finfra-shared","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwuxxin%2Finfra-shared/lists"}