{"id":18373118,"url":"https://github.com/thanatisia/distinstall-python","last_synced_at":"2025-04-06T19:32:20.093Z","repository":{"id":196661589,"uuid":"696670971","full_name":"Thanatisia/distinstall-python","owner":"Thanatisia","description":"A universal linux distribution installation framework/library that is focused on Modularity, Configurability, Portability and Readability. ","archived":false,"fork":false,"pushed_at":"2024-06-12T13:37:22.000Z","size":97643,"stargazers_count":7,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-22T05:51:08.216Z","etag":null,"topics":["archlinux","automation","distribution","distros","framework","installer","linux","programming","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Thanatisia.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOGS.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-09-26T08:02:33.000Z","updated_at":"2025-01-05T15:14:41.000Z","dependencies_parsed_at":null,"dependency_job_id":"c7414a2b-a2e9-4503-af6e-7fbd4f486c1d","html_url":"https://github.com/Thanatisia/distinstall-python","commit_stats":null,"previous_names":["thanatisia/distinstall-python"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Thanatisia%2Fdistinstall-python","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Thanatisia%2Fdistinstall-python/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Thanatisia%2Fdistinstall-python/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Thanatisia%2Fdistinstall-python/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Thanatisia","download_url":"https://codeload.github.com/Thanatisia/distinstall-python/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247539274,"owners_count":20955285,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["archlinux","automation","distribution","distros","framework","installer","linux","programming","python"],"created_at":"2024-11-06T00:08:44.454Z","updated_at":"2025-04-06T19:32:15.080Z","avatar_url":"https://github.com/Thanatisia.png","language":"Python","readme":"# py-unidist\n\n## Information\n### Description\n+ A Universal Distribution Install library/framework written in Python\n- This package contains\n    - working pre-made script for installing distributions\n        - Currently supported systems\n            + ArchLinux\n    + modules for installing/setting up the base/root filesystem of various distributions/systems using bootstrap installation\n\n+ ![py-distinstall full demo](resources/demo/demo-archlinux-full.gif)\n\n### Project\n+ Package Name: py-unidist\n+ Current Version: v0.4.5\n\n### Timeline\n+ The project has went through various major checkpoints and changes including rewriting from the original [Bash shellscript implementation - distro-installscript-arch](https://github.com/Thanatisia/distro-installscript-arch) as a standalone ArchLinux base filesystem installer via bootstrap installation\n+ In the midst of improving the project, I realised that the installation flow at times can be quite similar, it might work as a library/framework.\n- Additionally, at about that period, I felt that this deserves to be written in a proper programming language (such as Python, Golang, Rust etc) if it helps with scalability\n    + Because I was using Python more often during that period, I went with Python\n\n## Setup\n### Dependencies\n- pacman\n- parted\n- arch-install-scripts\n    + pacstrap\n+ dd\n+ dhcpcd\n+ python\n- python pypi packages\n    + ruamel.yaml : For YAML configuration file handling\n\n### Pre-Requisites\n- (Optional; Recommended) Setup python Virtual Environment\n    - Install dependencies\n        + venv/virtualenv\n    - Generate virtual environment\n        ```console\n        python -m venv [virtual-environment]\n        ```\n    - Chroot/Source the virtual environment\n        - *NIX\n            ```console\n            . [virtual-environment]/bin/activate\n            ```\n        - Windows\n            ```console\n            [virtual-environment]\\Scripts\\activate\n            ```\n\n- (Optional) Performing within a non-ArchLinux system\n    - Create a chroot environment with pacman/for ArchLinux\n        - Using 'archlinux/devtools'\n            - Dependencies\n                - [archlinux/devtools](https://gitlab.archlinux.org/archlinux/devtools) : Tools for Archlinux package maintainers\n                \n            - Make your chroot folder to store the chroot root environment\n                ```console\n                mkdir chroots\n                ```\n\n            - Create the actual chroot directory within it\n                - Using mkarchroot\n                    - Explanation\n                        - mkarchroot will \n                            + create the actual chroot environment - named 'root' - within the folder 'chroots'\n                            - Afterwhich, the packages and package groups specified - in this case, 'base' - will be bootstrapped and installed into the chroot environment's filesystem\n                                + 'base' is the package group that contains the root filesystem, this is necessary to make a working environment\n                    ```console\n                    mkarchroot chroots/root base base-devel git arch-install-scripts parted vim dhcpcd python3 python-pip python-ruamel-yaml\n                    ```\n\n            - Edit the mirrorlist within the chroot environment to facilitate the downgrade\n                - Explanation\n                    - Write the line 'Server = https://archive.archlinux.org/repos/[year]/[month]/[date]/$repo/os/$arch' into the mirrorlist of the chroot environment\n                        - where\n                            + year = the year of the repository; i.e. 2016\n                            + month = the month of the repository; i.e. 02 = February\n                            + date = the date of the repository; i.e. 19\n                ```console\n                echo 'Server = https://archive.archlinux.org/repos/[year]/[month]/[date]/$repo/os/$arch' \u003e [chroot-environment-path]/root/etc/pacman.d/mirrorlist\n                ```\n\n            - Enter the chroot environment\n                - Explanation\n                    + This is just basically chroot (Change root)-ing into the newly create ArchLinux filesystem chroot environment as per normal\n                    - However, in this example\n                        - We will be using 'arch-chroot' which can be found in the ArchLinux package 'arch-install-scripts' which is effectively a wrapper that \n                            + performs several mounts before chroot is ran\n                ```console\n                sudo arch-chroot [chroot-environment-path]/root [shell]\n                ```\n\n        - Using docker\n            - Setup\n                - Dependencies\n                    + docker\n                    + (Optional) docker-compose\n                - Add your user into the 'docker' group to be able to use docker-compose and docker functionalities\n                    ```console\n                    sudo usermod -aG docker [username]\n                    ```\n                - Restart your system to refresh the permission\n                    ```console\n                    sudo reboot now\n                    ```\n            - Pull latest archlinux image\n                ```console\n                docker pull archlinux:latest\n                ```\n            - (Optional) Use the Dockerfile created with the proper packages pre-defined\n                - Information\n                    + Located in the folder [docker](docker)\n                - Build the Dockerfile of your choice\n                    ```console\n                    docker build -t [tag] -f docker/[dockerfile-of-choice] [context-directory]\n                    ```\n            - Startup a container\n                - Using docker run\n                    - Explanation\n                        - Startup an ArchLinux docker container with the name 'arch-chroot'\n                            - Enable privileged mode so that \n                                - the device can be passthrough from host into the container and\n                                - the passthrough devices can be modified, and used\n                            - Adding/Passthrough the disk/devices to container\n                                - Examples\n                                    - SATA/AHCI\n                                        ```console\n                                        --device=/dev/sdX\n                                        ```\n                                    - NVME\n                                        ```console\n                                        --device=/dev/nvme[device-number]\n                                        ```\n                                    - Loopback Device\n                                        ```console\n                                        --device=/dev/loop[device-number]\n                                        ```\n                                ```console\n                                --device=[disk-label]\n                                ```\n                            - Mount the following volumes from the host system to the container\n                                - [host-system-source-volume] =\u003e [container-destination-volume]\n                                    - Mounting disk/devices from host system into container\n                                        - Use-Case:\n                                            + Useful for using the container rootfs environment as a temporary chroot environment to install a distribution via command line using bootstrapping (ArchLinux style)\n                                        + Structure: `[host-target-disk-label] =\u003e [container-disk-label]`\n                                        - (Optional) Mount distinstall-python repository from the host system to the container (if it already exists and you want to use that)\n                                            ```console\n                                            -v \"/path/to/distinstall-python:/tmp/distinstall-python\"\n                                            ```\n                            - Set the network mode to \"host\"\n                                - So that the container is using the host's network address for consistency whilst formatting and bootstrapping\n                        - Parameters\n                    ```console\n                    docker run -itd --name=arch-chroot --privileged --device=[disk-label] \"/path/to/distinstall-python:/tmp/distinstall-python\" --network=host {other-options} archlinux:latest\n                    ```\n            - Chroot (Change root) into the container\n                ```console\n                docker exec -it [container-name] [shell]\n                ```\n            - (Optional) If you are using the default official image(s)\n                - ArchLinux\n                    ```console\n                    docker exec -it [container-name] /bin/bash -c \"pacman -Syu \u0026\u0026 pacman -S base-devel git arch-install-scripts parted vim dhcpcd python3 python-pip python-ruamel-yaml\"\n                    ```\n                - Debian\n                    ```console\n                    docker exec -it [container-name] /bin/bash -c \"apt update -y \u0026\u0026 apt upgrade -y \u0026\u0026 apt install -y base-devel git arch-install-scripts parted vim dhcpcd python3 python-pip python-ruamel-yaml\"\n                    ```\n            - After initial startup \n                - Explanation\n                    + If you did not remove the container, you can just start the container back up after every restart if you require the chroot environment\n                - Start up the container\n                    ```console\n                    docker start [container-name]\n                    ```\n\n        - In the chroot environment\n            - Update accordingly\n                ```console\n                pacman -Syu\n                ```\n\n- Install system dependencies\n    - From pkglist\n        - If using pacman\n            ```console\n            pacman -S - \u003c pkglist.txt\n            ```\n        - If using apt\n            ```console\n            apt install \u003c pkglist.txt\n            ```\n\n- (Optional) If you are installing into a Virtual Disk Image on Host\n    - Create a Virtual Hard Disk image\n        - Explanation\n            - Parameters\n                - Positionals\n                    - path-to-virtual-hard-disk-img : This specifies the output file to create\n                - Options\n                    - bs : This contains the size (in bytes) of each block that dd will read each time\n                    - count : This specifies the number of counts to write each block in the Virtual Disk\n            - Therefore, to create a VHD of size 56gb\n                - Options\n                    - bs = 1G\n                    - count = 56\n                - 1G * 56 = 56Gb\n        ```console\n        dd if=/dev/zero of=[path-to-virtual-hard-disk-img] bs=[block-size (bytes-per-block)] count=[number-of-counts]\n        ```\n    - Mount Disk Image and Partitions as loopback devices\n        - Using losetup\n            - Pre-Requisites\n                + losetup\n            - Explanation\n                - Parameters\n                    + -P\n                    + -f : Print the first available loop device\n            ```console\n            sudo losetup -Pf [path-to-virtual-hard-disk-img]\n            ```\n\n### Installation\n- Using python pip\n    - Install git package\n        - From GitHub\n            ```bash\n            python3 -m pip install git+https://github.com/Thanatisia/py-unidist{@[branch-tag-name]}\n            ```\n        - From requirements.txt\n            - Prepare requirements.txt file\n                ```\n                distinstall-python @ git+https://github.com/Thanatisia/py-unidist{@[branch-tag-name]}\n                ```\n            - (Optional) Install requirements.txt\n                ```bash\n                python3 -m pip install -Ur requirements.txt\n                ```\n\n    - Manually via cloning\n        - Clone git repository\n            ```bash\n            git clone https://github.com/Thanatisia/py-unidist\n            ```\n        - Change directory into project root\n            ```bash\n            cd py-unidist\n            ```\n        - (Optional) Install python dependencies\n            ```bash\n            python3 -m pip install -Ur requirements.txt\n            ```\n        - Install package as development mode\n            ```bash\n            pip install .\n            ```\n        - (Optional) Uninstall package\n            ```bash\n            pip uninstall py-unidist\n            ```\n\n### Compiling into an executable\n\u003e Still undergoing tests\n- Using PyInstaller\n    - Pre-Requisites\n        - Dependencies\n            - Python package\n                + pyinstaller\n    - Compile/Build the executable\n        - Parameters\n            + --onefile : Generate a single executable file with everything bundled inside\n        - Notes\n            - Output: \n                + dist: The executable will be placed here\n        - Syntax\n            ```console\n            {python -m} pyinstaller --onefile [main-driver-file]\n            ```\n        - Usage\n            ```console\n            {python -m} pyinstaller --onefile main.py\n            ```\n\n### Quickstart Flow\n#### Users\n1. Perform setup\n    - Create Virtual Environment\n        ```console\n        python3 -m venv [virtual-environment-name]\n        ```\n    - Source and chroot into Virtual Environment\n        - Linux\n            ```bash\n            . [virtual-environment-name]/bin/activate\n            ```\n        - Windows\n            ```bash\n            .\\[virtual-environment-name]\\Scripts\\activate\n            ```\n    - Install dependencies\n        ```console\n        pip install -Ur requirements.txt\n        ```\n1. Installing package\n    + Follow the steps in [Installation](#installation) to install the package\n\n2. Generate configuration file\n    ```console\n    py-distinstall --generate-config\n    ```\n3. Edit configuration file\n    + Please refer to [USAGE.md](USAGE.md) for more information on customization\n    + Under the 'customization' section\n4. Perform a test run\n    - Explanation\n        + The Default mode is 'Debug', this will print the commands only\n    - Perform full start\n        ```console\n        sudo py-distinstall start\n        ```\n    - Execute specific stages\n        ```console\n        sudo py-distinstall --execute-stage [stage-number]\n        ```\n5. Once you have confirmed\n    - Explanation\n        + To run the changes proper, you need to set the mode to RELEASE\n    - Perform full start\n        ```console\n        sudo py-distinstall --mode RELEASE start\n        ```\n    - Execute specific stages\n        ```console\n        sudo py-distinstall --mode RELEASE --execute-stage [stage-number]\n        ```\n\n#### Developers\n- To implement a new distribution installation step\n    - Make directory for the new base distribution\n        ```bash\n        mkdir -pv src/pydistinstall/app/distributions/[new-base-distribution-name]\n        ```\n    - Copy the template 'mechanism.py' from any of the 'src/pydistinstall/app/distributions/[base-distribution-name]' to a test draft location\n        ```bash\n        cp -r src/pydistinstall/app/distributions/[base-distribution-name] src/pydistinstall/app/distributions/[new-base-distribution-name]\n        ```\n    - Edit accordingly\n        + Please ensure that to maintain consistency, function names is adviced to be the same if the function already exists.\n        + If the function didnt exist, then you may create a new function to be called as necessary\n\n## Documentation\n### Synopsis/Syntax\n- Basic Run\n    ```console\n    {sudo} py-distinstall {options} \u003carguments\u003e [positionals]\n    ```\n\n### Parameters\n- Positionals\n- Optionals\n    - With Arguments\n        + `-c [config-file-name] | --config      [config-file-name]` : Set custom configuration file name\n        + `-d [target-disk-name] | --target-disk [target-disk-name]` : Set target disk name\n        + `-e [default-editor]   | --editor      [default-editor]`   : Set default text editor\n        + `-m [DEBUG|RELEASE]    | --mode        [DEBUG|RELEASE]`    : Set mode (DEBUG|RELEASE)\n        + `-u [mount-point]      | --unmount     [mount-point]`      : Unmount the drive from the mount points specified in the configuration file\n        + `--execute-stage [stage-number]`                           : Specify an installation stage number to execute\n    - Flags\n        + --display-options         : Display all options\n        + -g | --generate-config    : Generate configuration file\n        + --print-config            : Import configuration file, load it and print contents\n        + -h | --help               : Display this help menu and all commands/command line arguments\n        + --fdisk                   : Open up fdisk for manual partition configuration\n        + --cfdisk                  : Open up cfdisk for manual partition configuration\n        + --list-stages             : List all installation steps/stages of the target platform to install\n        + -v | --version            : Display system version information\n        + start                     : Start the full base installer + post-installer process\n\n### Modes\n+ DEBUG (Default) : Test install; Allows you to see all the commands that will be executed if you set the MODE to 'RELEASE'; set by default to prevent accidental reinstallation/overwriting\n+ RELEASE : Performs the real RELEASE; must use with sudo\n\n### Environment Variables\n+ TARGET_DISK_NAME : This is used in the environment variable to specify the target disk you want to install with\n+ MODE : This indicates the execution permission of the application; Default: DEBUG; Set this to 'RELEASE' to begin and commit execution and changes to be made\n\n### Usage\n- Generate configuration file\n    ```console\n    py-distinstall --generate-config\n    ```\n\n- Default (Test Install; Did not specify target disk name explicitly)\n    ```console\n    sudo py-distinstall start\n    ```\n\n- Unmount the drive from the mount points specified in the configuration file\n    ```console\n    sudo py-distinstall -u [root-mount-point]\n    ```\n\n- Test Install; with target disk name specified as flag\n    ```console\n    sudo py-distinstall -d \"/dev/sdX\" start\n    ```\n\n- Test Install; with target disk name specified with environment variable TARGET_DISK_NAME\n    ```console\n    sudo TARGET_DISK_NAME=\"/dev/sdX\" py-distinstall start\n    ```\n\n- Test Install; with custom configuration file\n    ```console\n    sudo py-distinstall -c \"new config file\" -d \"/dev/sdX\" start\n    ```\n\n- Start installation (Did not specify target disk name explicitly)\n    ```console\n    sudo py-distinstall -m RELEASE start\n    ```\n\n- Start installation with the start mode specified with environment variable 'MODE'\n    ```console\n    sudo MODE=RELEASE py-distinstall start\n    ```\n\n- Start installation (with target disk name specified as flag)\n    ```console\n    sudo py-distinstall -d \"/dev/sdX\" -m RELEASE start\n    ```\n\n- Start installation (with target disk name specified with environment variable TARGET_DISK_NAME)\n    ```console\n    sudo TARGET_DISK_NAME=\"/dev/sdX\" py-distinstall -m RELEASE start\n    ```\n\n- Start installation (with custom configuration file)\n    ```console\n    sudo py-distinstall -c \"new config file\" -d \"/dev/sdX\" -m RELEASE start\n    ```\n\n- List all installation stages\n    ```console\n    sudo py-distinstall --list-stages\n    ```\n\n- Execute specific stages\n    ```console\n    sudo py-distinstall --execute-stage 1 --execute-stage 2 .... -m RELEASE\n    ```\n\n- Open up fdisk for manual partition configuration\n    ```console\n    sudo py-distinstall --fdisk\n    ```\n\n- Open up fdisk for manual partition configuration\n    ```console\n    sudo py-distinstall --cfdisk\n    ```\n\n### Notes\n+ For more information on usage and customization, please refer to [USAGE.md](USAGE.md)\n\n## Wiki\n\n### Project Structure\n```\nproject-root/\n    |\n    |-- src/ : For all project-related files\n        |\n        |-- pydistinstall/ : Main package\n            |\n            |-- main.py  : The main runner/launcher project code\n            |-- setup.py : Root setup file for the main runner/launcher\n            |-- unittest.py : WIP Unit Testing source file\n            |-- app/ : For all application-specific functionalities; Such as source files related to the installation mechanism of the various Distributions\n                |\n                |-- runner.py : This is the Distribution Switcher (\"Load Balancer\") that will process your target distribution name and separate to the appropriate distributions\n                |-- distributions/ : For all distribution classes\n                    |\n                    |-- archlinux/ : Contains ArchLinux installation functionality and archlinux-specific libraries\n                        |\n                        |-- mechanism.py : The primary library containing the Base Installation and Post-Installation mechanism classes for the distribution\n            |-- lib/ : For all external/general libraries that are not application-specific\n                |\n                |-- cli.py : This contains functionality to Command Line Interface (CLI) Argument handling\n                |-- config_handler.py : This contains functionality to handling Configuration Files\n                |-- const.py : This contains constant variables and values\n                |-- device_management.py : This contains Device and Disk Handling functions\n                |-- env.py : This contains Environment Variables\n                |-- format.py : This contains string formatting support\n                |-- process.py : This contains Subprocess and systems command execution functions\n                |-- user_management.py : This contains User management functionalities\n                |-- utils.py : This contains general utilities\n```\n\n## TODO\n### Pipeline\n+ [ ] Migration from Linux Bash Shellscript to Python\n- [ ] Configuration File Handling and Support\n    - [ ] Key-Values\n        - [X] user_ProfileInfo\n            - [X] Change ['secondary_Groups'] into a list instead of a standalone\n        - [X] Plan to rename \"device_Type\" into \"storage_Controller\" for a better, more accurate name, OR to add a separate group called \"storage_Controller\"\n        - [ ] Change configuration naming scheme\n    - [ ] Support for JSON\n    - [X] Support for segmented running - Running only specific steps at any one time\n    - Support for different disk medium typings\n        - [X] NVME : /dev/nvme[disk-number]p[partition-number]\n        - [X] Loopback device : /dev/loop[loopback-number]p[partition-number]\n- [ ] Planned Quality-of-Life changes\n    - [ ] Improved Readability\n        + [ ] Usage of proper data structure objects such as Dictionary for Key-value mappings and Lists for Arrays and iterative data objects\n        + [ ] Rename YAML configuration file keyword naming convention\n    - [ ] Improved portability, customizability and modularity\n        + [ ] Configuration File I/O: Using YAML serialized data object for configuration file handling instead of shellscript sourcing (and potentially classic JSON support, as well as other configuration file types if enough support and works)\n        + [ ] Convenience: Easier to perform rewrites (if necessary)\n        + [ ] Easier distribution methods - i.e. Compilation into binary using compilers for python like pyinstaller\n- [ ] Bug Fixes\n    + [ ] Fixed technical terminologies and makes it easier to understand\n- [ ] Refactoring\n    + Remake the main CLI utility entry point to be less messy\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthanatisia%2Fdistinstall-python","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthanatisia%2Fdistinstall-python","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthanatisia%2Fdistinstall-python/lists"}