{"id":24574769,"url":"https://github.com/johndeedly/devops-linux","last_synced_at":"2025-03-17T11:41:37.372Z","repository":{"id":273563394,"uuid":"918983748","full_name":"johndeedly/devops-linux","owner":"johndeedly","description":"Deploy cloud ready images in virtual machines, container or on metal via .iso or pxe boot","archived":false,"fork":false,"pushed_at":"2025-03-16T06:06:46.000Z","size":733,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-16T07:19:14.875Z","etag":null,"topics":["archlinux","bare-metal","container","debian","deploy","devops","docker","iso","linux","pxe-boot","rockylinux","ubuntu","virtual-machine"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"0bsd","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/johndeedly.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-19T12:10:47.000Z","updated_at":"2025-03-16T06:06:49.000Z","dependencies_parsed_at":"2025-01-21T17:29:47.099Z","dependency_job_id":"ad08b2bc-eefe-4e67-8b42-170b4502d82d","html_url":"https://github.com/johndeedly/devops-linux","commit_stats":null,"previous_names":["johndeedly/devops-linux"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/johndeedly%2Fdevops-linux","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/johndeedly%2Fdevops-linux/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/johndeedly%2Fdevops-linux/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/johndeedly%2Fdevops-linux/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/johndeedly","download_url":"https://codeload.github.com/johndeedly/devops-linux/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244029486,"owners_count":20386416,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["archlinux","bare-metal","container","debian","deploy","devops","docker","iso","linux","pxe-boot","rockylinux","ubuntu","virtual-machine"],"created_at":"2025-01-23T21:31:03.727Z","updated_at":"2025-03-17T11:41:37.367Z","avatar_url":"https://github.com/johndeedly.png","language":"Shell","readme":"# DevOps Linux\n\u003cdiv style=\"text-align: center;\"\u003e\n    \n![Linux server in a container](devops.jpg)\n\n\u003c/div\u003e\n\n## Repository\n\nThis source code is available to everyone under the standard [0BSD License](LICENSE.txt) to allow every setup configuration in every way possible, commercial background or not. Some of the scripts in this project **will** destroy all data on your system. So be careful and use a testing lab first! Common sense, people. **I will not take any responsibility for any of your lost files!**\n\n## Build Environment\n\nYour build environment should include the following programs:\n- **packer** for automation,\n- **swtpm** for TPM emulation,\n- **cloud-image-utils** to package the complete build chain into a single mime-multipart user-data file,\n- **xorriso** to repackage the Arch ISO,\n- **yq** for config file parsing, and\n- **qemu-desktop** (Arch) / **qemu-system-x86_64** (Debian/Ubuntu) for virtualization.\n\nThe project has the following folder structure:\n- **📁build** - in this folder all the files are prepared before they are placed inside the Arch ISO\n- **📁config** - the main folder containing all the python, setup and config files\n- **📁config/📄setup.yml** - central configuration file describing all the files that are needed for the installation\n- **📁database** - a temporary folder for package caching, allowing for installations where no internet is available to use a local package cache from previous installations\n- **📁output** - the final artifacts are placed inside this folder\n- **📁output/📁artifacts/📁docker** - the produced docker image is placed here\n- **📁output/📁artifacts/📁pxe** - the produced files for pxe booting are placed here\n- **📁output/📁devops-linux** - the produced virtual machine is placed here\n- **📁output/📁devops-linux/📄devops-linux-x86_64.run.sh** - the main executable script for the produced virtual machine\n- **📄cidata.sh** - preparation script to package the files needed for CIDATA execution of cloud-init\n- **📄pipeline.sh** - this script will start the whole setup pipeline\n\nSupported cloud images are Arch, Ubuntu, Debian and Rocky Linux, although Rocky is not well tested, as I mainly utilize Arch for clients and Debian for servers.\n\n## Config File Structure\n\n**📁config/📄setup.yml**\n```yaml\n## Mapping distros to their packaging tools (debian -\u003e apt, rocky -\u003e yum, ...)\ndistros:\n  [...]\n## Where to download the corresponding qcow2 image\n## \"archiso\" is an exception as the entry maps to the Arch ISO download link\ndownload:\n  [...]\n## End of life for all the distro package versions. Will print an error and abort the setup\n## when no support is to be expected from the maintainers any more.\nendoflife:\n  [...]\n## The file name of the downloaded image\nimages:\n  [...]\n## setup files per stage and packaging tool\n## format: [packaging tool] -\u003e [setup name] -\u003e [path/stage/config file]\nfiles:\n  [...]\n## setup instructions\nsetup:\n  ## chosen distro name\n  distro: [...]\n  ## which files to install via setup name and distro\n  options:\n    - [...]\n  ## legacy entry to define environment variables. I'm not sure whether it should be marked deprecated or not.\n  env_vars:\n    - [...]\n  ## the path to the target device to write the cloud image onto. \"auto\" tries to find a hard drive on it's own, but errors out when nothing is found.\n  target: auto\n  ## (local) mirror link to the base of archive.archlinux.org\n  archiso_mirror: false\n  ## (local) package mirror link\n  pkg_mirror: false\n  ## (local) Chaotic Arch user repository mirror link\n  chaotic_mirror: false\n```\n\nAt this point the following options can be selected for installation:\n\n- **base** (_all_): Some programs installed for ease of use, performing most of the tasks inside the terminal. The graphics drivers for the most common (virtual) GPUs are placed here, too. They are not needed for the essential basics.\n- **mirror** (_all_): Local mirrors for all supported distributions, downloading updates every couple of days for all the packages while keeping the last two or more versions accessible. Instead of using the official default route, we iterate over all available packages, retrieve the download urls and instead of using apt or pacman for the job we download all packages through wget, allowing to timestamp every file and force \"304 Not Modified\" messages. This method is so much faster and more efficient than the throttled and overrun rsync connections and allows for local repositories that are not officially supported by the vendor.\n- **cinnamon** (_Arch_): Install the fully configured [cinnamon](https://github.com/linuxmint/cinnamon) desktop, including graphical tools like office programs, video players, etc.\n- **kde** (_Arch_): Install the fully configured [kde plasma](https://kde.org/de/plasma-desktop/) desktop, including some default kde applications, excluding unneeded utilities like \"plasma-welcome\", \"kongress\", \"kteatime\" and such.\n- **podman** (_Arch, Debian, Ubuntu_): A docker replacement, that is fully compatible with all commands and hubs, and the new modern way to handle containers. In addition, [portainer](https://www.portainer.io/) is installed for easy container management via browser.\n- **postgres** (_Arch, Debian, Ubuntu_): Install [postgres](https://www.postgresql.org/) as a container.\n- **cronicle** (_Arch, Debian, Ubuntu_): Install [cronicle](https://github.com/jhuckaby/Cronicle) as a container.\n- **dagu** (_Arch, Debian, Ubuntu_): Install [dagu](https://github.com/dagu-org/dagu) as a container.\n- **plex** (_Arch, Debian, Ubuntu_): Install [plex media server](https://www.plex.tv/) as a container including shared gpu passthrough and configuration of the host.\n- **minecraft** (_Arch, Debian, Ubuntu_): Install [minecraft](https://www.minecraft.net/de-de) as a container.\n- **gitlab** (_Arch, Debian, Ubuntu_): Install [gitlab](https://about.gitlab.com/) as a container.\n- **router** (_Arch_): A fully functional virtual router with DHCP4, DHCP6, DNS, NTP, PXE boot and ACME certificate authority. Connect the router through ```-netdev socket,listen=...``` with subsequent virtual machines ```-netdev socket,connect=...```. To arm PXE boot with the prebuild initramfs, kernel and image, the ```📁output/📁artifacts``` folder in the default configuration can be mounted via ```mount -t 9p artifacts.0 /mnt``` and the contents then copied to ```cp /mnt/pxe/arch/x86_64/* /srv/pxe/arch/x86_64/```.\n- **proxmox** (_Debian_): Install [proxmox](https://www.proxmox.com/en/) to configure and spawn virtual machines and LXC container via gui.\n- **podman-image** (_Arch, Debian_): As the final step, take everything that was configured before and generate a fully functional OCI container, that can be uploaded to any docker or podman instance.\n- **pxe-image** (_Arch, Debian_): As the final step, take everything that was configured before and generate a fully functional pxe boot image, that can e.g. be used in conjunction with the router option above to netboot any device on the LAN. The Arch PXE image is able to be booted via CIFS, HTTP, ISCSI, NBD, NFS, NVMEOF and SCP (never saw that one before, right? 😉).\n\n## Common Setups (by me)\n\n### Setup #1: PXE image with kde and basic tools\n```yaml\nsetup:\n  distro: archlinux\n  options:\n    - base\n    - kde\n    - pxe-image\n```\n\n### Setup #2: Router to host the pxe image produced by #1 for testing\n```yaml\nsetup:\n  distro: archlinux\n  options:\n    - base\n    - router\n```\n\n### Setup #3: Proxmox server\n```yaml\nsetup:\n  distro: debian\n  options:\n    - base\n    - proxmox\n```\n\n### Setup #4: Archlinux and Debian local mirror\n```yaml\nsetup:\n  distro: archlinux\n  options:\n    - mirror\n\nsetup:\n  distro: debian\n  options:\n    - mirror\n```\n\n### Setup #5: Build a minimal Arch Linux or Debian container \n```yaml\nsetup:\n  distro: archlinux\n  options:\n    - podman-image\n\nsetup:\n  distro: debian\n  options:\n    - podman-image\n```\n\n### Setup #6: Podman server to host dagu for automation\n```yaml\nsetup:\n  distro: debian\n  options:\n    - base\n    - podman\n    - dagu\n```\n\n## License\n\nLicensed under the [0BSD](LICENSE.txt) license.","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjohndeedly%2Fdevops-linux","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjohndeedly%2Fdevops-linux","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjohndeedly%2Fdevops-linux/lists"}