{"id":13753597,"url":"https://github.com/openwisp/ansible-openwisp2-imagegenerator","last_synced_at":"2025-04-05T12:06:29.551Z","repository":{"id":13909934,"uuid":"74664316","full_name":"openwisp/ansible-openwisp2-imagegenerator","owner":"openwisp","description":"Automatically build several openwisp2 firmware images for different organizations while keeping track of their differences","archived":false,"fork":false,"pushed_at":"2025-03-06T20:01:16.000Z","size":107,"stargazers_count":120,"open_issues_count":2,"forks_count":14,"subscribers_count":16,"default_branch":"master","last_synced_at":"2025-03-29T11:08:34.669Z","etag":null,"topics":["access-point","ansible","ansible-role","hacktoberfest","networking","openwrt","router","wifi","wireless"],"latest_commit_sha":null,"homepage":"http://openwisp.org","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/openwisp.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-11-24T10:50:41.000Z","updated_at":"2025-03-06T20:01:21.000Z","dependencies_parsed_at":"2024-09-25T00:30:20.423Z","dependency_job_id":null,"html_url":"https://github.com/openwisp/ansible-openwisp2-imagegenerator","commit_stats":{"total_commits":95,"total_committers":7,"mean_commits":"13.571428571428571","dds":0.3263157894736842,"last_synced_commit":"ad65c7ca64a84fafffc96461a058323cefac9ecb"},"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openwisp%2Fansible-openwisp2-imagegenerator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openwisp%2Fansible-openwisp2-imagegenerator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openwisp%2Fansible-openwisp2-imagegenerator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openwisp%2Fansible-openwisp2-imagegenerator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/openwisp","download_url":"https://codeload.github.com/openwisp/ansible-openwisp2-imagegenerator/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247332605,"owners_count":20921853,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["access-point","ansible","ansible-role","hacktoberfest","networking","openwrt","router","wifi","wireless"],"created_at":"2024-08-03T09:01:25.483Z","updated_at":"2025-04-05T12:06:29.518Z","avatar_url":"https://github.com/openwisp.png","language":"Shell","funding_links":[],"categories":["networking"],"sub_categories":[],"readme":"ansible-openwisp2-imagegenerator\n================================\n\n[![Galaxy](http://img.shields.io/badge/galaxy-openwisp.openwisp2--imagegenerator-blue.svg?style=flat-square)](https://galaxy.ansible.com/openwisp/openwisp2-imagegenerator/)\n\nThis ansible role allows to build several openwisp2 firmware images for different organizations while\nkeeping track of their configurations.\n\n**NOTE**: this role has not been tested in the wild yet.\nIf you intend to use it, try it out and if something goes wrong please proceed to report your issue,\nbut bear in mind you need to be willing to understand the [build process](#build-process)\nand it's inner working in order to make it work for you.\n\nRequired role variables\n=======================\n\nThe following variables are required:\n\n* `openwisp2fw_source_dir`: indicates the directory of the [OpenWrt](https://openwrt.org/) source that is used during [compilation](#2-compilation)\n* `openwisp2fw_generator_dir`: indicates the directory used for the [preparation of generators](#3-preparation-of-generators)\n* `openwisp2fw_bin_dir`: indicates the directory used when [building the final images](#4-building-of-final-images)\n* `openwisp2fw_organizations`: a list of organizations; see the example `playbook.yml` file in the\n  [create playbook section](#5-create-playbook-file) to understand its structure\n\nUsage (tutorial)\n================\n\nIf you don't know how to use ansible, don't panic, this procedure will guide you step by step.\n\nIf you already know how to use ansible, you can skip this section and jump straight to the\n\"Install this role\" section.\n\nFirst of all you need to understand two key concepts:\n\n* for **\"compilation server\"** we mean a server which is used to compile the images\n* for **\"local machine\"** we mean the host from which you launch ansible, eg: your own laptop or CI server\n\nAnsible is a configuration management/automation tool that works by entering the compilation server\nvia SSH and executing a series of commands.\n\n### 1. Install ansible\n\nInstall ansible **on your local machine** if you haven't done already, there are various ways\nin which you can do this, but we prefer to use the official python package manager, eg:\n\n    sudo pip install ansible\n\nIf you don't have pip installed see [Installing pip](https://pip.pypa.io/en/stable/installing/)\non the pip documentation website.\n\n[Installing ansible in other ways](http://docs.ansible.com/ansible/intro_installation.html#latest-release-via-yum)\nis fine too, just make sure to install a version of the `2.0.x` series (which is the version with\nwhich we have tested this playbook).\n\n### 2. Install this role\n\nFor the sake of simplicity, the easiest thing is to install this role **on your local machine**\nvia `ansible-galaxy` (which was installed when installing ansible), therefore run:\n\n    sudo ansible-galaxy install openwisp.openwisp2-imagegenerator\n\n### 3. Choose a working directory\n\nChoose a working directory **on your local machine** where to put the configuration of\nyour firmware images.\n\nEg:\n\n    mkdir ~/my-openwisp2-firmware-conf\n    cd ~/my-openwisp2-firmware-conf\n\nPutting this working directory under version control is also a very good idea, it will help you to\ntrack change, rollback, set up a CI server to automatically build the images for you and so on.\n\n### 4. Create inventory file\n\nThe inventory file is where group of servers are defined. In our simple case we can get away with\ndefining a group in which we will put just one server.\n\nCreate a new file `hosts` **on your local machine** with the following contents:\n\n    [myserver]\n    mycompiler.mydomain.com ansible_user=\u003cyouruser\u003e ansible_become_pass=\u003csudo-password\u003e\n\nSubstitute `mycompiler.mydomain.com` with your hostname (ip addresses are allowed as well).\n\nAlso put your SSH user and password respectively in place of `\u003cyouruser\u003e` and `\u003csudo-password\u003e` (must be sudoer and non-root).\nThese credentials are used during the [Installation of dependencies step](#1-installation-of-dependencies).\n\n### 5. Create playbook file\n\nCreate a new playbook file `playbook.yml` **on your local machine** with the following contents:\n\n```yaml\n# playbook.yml\n- hosts: your_host_here\n  roles:\n    - openwisp.openwisp2-imagegenerator\n  vars:\n    openwisp2fw_source_dir: /home/user/openwisp2-firmware-source\n    openwisp2fw_generator_dir: /home/user/openwisp2-firmware-generator\n    openwisp2fw_bin_dir: /home/user/openwisp2-firmware-builds\n    openwisp2fw_source_targets:\n        - system: ar71xx\n          subtarget: generic\n          profile: Default\n        - system: x86\n          subtarget: generic\n          profile: Generic\n    openwisp2fw_organizations:\n        - name: snakeoil # name of the org\n          flavours: # supported flavours\n            - standard\n          luci_openwisp: # /etc/config/luci_openwisp\n            # other config keys can be added freely\n            username: \"operator\"\n            # clear text password that will be encrypted in /etc/config/luci_openwisp\n            password: \"\u003cCHANGE_ME\u003e\"\n          openwisp: # /etc/config/openwisp\n            # other config keys can be added freely\n            url: \"https://my-openwisp2-instance.com\"\n            shared_secret: \"my-openwisp2-secret\"\n            unmanaged: \"{{ openwisp2fw_default_unmanaged }}\"\n          # clear text password that will be encrypted in /etc/shadow\n          root_password: \"\u003cCHANGE_ME\u003e\"\n```\n\nThis playbook will let you compile firmware images for an organization named `snakeoil` using only\nthe `standard` flavour (which includes a default OpenWrt image with the standard OpenWISP2 modules)\nfor two architectures, ar71xx and x86.\n\nSee the section [Role Variables](#role-variables) to know how to customize the available configurations.\n\nAt this stage your directory layout should look like the following:\n\n```\n.\n├── hosts\n└── playbook.yml\n```\n\n### 6. Run the playbook\n\nNow is time to **start the compilation of OpenWISP2 Firmware**.\n\nLaunch the playbook **from your local machine** with:\n\n    ansible-playbook -i hosts playbook.yml -e \"recompile=1 cores=4\"\n\nYou can substitute `cores=4` with the number of cores at your disposal.\n\nWhen the playbook is done running you will find the images on **the compilation server** located in\nthe directory specified in `openwisp2fw_bin_dir`, which in our example is\n`/home/user/openwisp2-firmware-builds` with a directory layout like the following:\n\n```\n/home/user/openwisp2-firmware-builds\n├── snakeoil/                                      # (each org has its own dir)\n├── snakeoil/2016-12-02-094316/ar71xx/standard/    # (contains ar71xx images for standard flavour)\n├── snakeoil/2016-12-02-094316/x86/standard/       # (contains x86 images for standard flavour)\n└── snakeoil/latest/                               # (latest is a symbolic link to the last compilation)\n```\n\nNow, if you followed this tutorial and everything worked out, you are ready to customize your\nconfiguration to suit your needs! Read on to find out how to accomplish this.\n\nRole variables\n==============\n\nThere are many variables that can be customized if needed, take a look at\n[the defaults](https://github.com/openwisp/ansible-openwisp2-imagegenerator/blob/master/defaults/main.yml)\nfor a comprehensive list.\n\nSome of those variables are also explained in [Organizations](#organizations) and [Flavours](#flavours).\n\nOrganizations\n=============\n\nIf you are working with OpenWISP, there are chances you may be compiling images for different groups\nof people: for-profit clients, no-profit organizations or any group of people that can\nbe defined as an \"*organization*\".\n\nOrganizations can be defined freely in `openwisp2fw_organizations`.\n\nFor an example of how to do this, refer to the \"[example playbook.yml file](#5-create-playbook-file)\".\n\nIf you need to add specific files in the filesystem tree of the images of each organization, see\n\"[Adding files for specific organizations](#adding-files-for-specific-organizations)\".\n\nFlavours\n========\n\nA flavour is a combination of packages that are included in an image.\n\nYou may want to create different flavours for your images, for example:\n\n* `standard`: the most common use case\n* `minimal`: an image for device which have little storage space available\n* `mesh`: an image with packages that are needed for implementing a mesh network\n\nBy default only a `standard` flavour is available.\n\nYou can define your own flavours by setting `openwisp2fw_image_flavours` - take a look at\n[the default variables](https://github.com/openwisp/ansible-openwisp2-imagegenerator/blob/master/defaults/main.yml)\nto understand its structure.\n\nBuild process\n=============\n\nThe build process is composed of the following steps.\n\n### 1. Installation of dependencies\n\n**Tag**: `install`.\n\nIn this phase the operating system dependencies needed for the subsequent steps are installed or upgraded.\n\n### 2. Compilation\n\n**Tag**: `compile`.\n\nThe OpenWrt source is compiled in order to produce something\ncalled \"[Image Generator](https://openwrt.org/docs/guide-user/additional-software/imagebuilder)\".\nThe *image generator* is an archive that contains the precompiled packages and a special\n`Makefile` that will be used to generate the customized images for each organization.\n\nThe source is downloaded and compiled in the directory specified in\n`openwisp2fw_source_dir`.\n\n### 3. Preparation of generators\n\n**Tags**: `extract`, `generator` (or `files`).\n\nDuring these steps the *image generators* are extracted and prepared\nfor building different images for different [organizations](#organizations), each organization\ncan build images for different [flavours](#flavours) (eg: full-featured, minimal, mesh, ecc);\n\nThe images are extracted and prepared in the directory specified in\n`openwisp2fw_generator_dir`.\n\n### 4. Building of final images\n\n**Tag**: `build`.\n\nIn this phase a series of images is produced.\n\nSeveral images will be built for each architecture, organization and flavour.\nThis can generate quite a lof of files, therefore use this power with caution: it's probably\nbetter to start with fewer options and add more cases as you go ahead.\n\nFor example, if you choose to use 2 architectures (ar71xx and x86), 2 organizations (eg: A and B)\nand 2 flavours (eg: standard and mini), you will get 8 groups of images:\n\n* **organization**: A / **flavour**: standard / **arch**: ar71xx\n* **organization**: A / **flavour**: standard / **arch**: x86\n* **organization**: A / **flavour**: mini / **arch**: ar71xx\n* **organization**: A / **flavour**: mini / **arch**: x86\n* **organization**: B / **flavour**: standard / **arch**: ar71xx\n* **organization**: B / **flavour**: standard / **arch**: x86\n* **organization**: B / **flavour**: mini / **arch**: ar71xx\n* **organization**: B / **flavour**: mini / **arch**: x86\n\nThe images will be created in the directory specified in\n`openwisp2fw_bin_dir`.\n\n### 5. Upload images to OpenWISP Firmware Upgrader\n\n**Tag**: `upload`.\n\nThe last step is to upload images to the\n[OpenWISP Firmware Upgrader module](https://github.com/openwisp/openwisp-firmware-upgrader).\nThis step is optional and disabled by default.\n\nTo enable this feature, the variables ``openwisp2fw_uploader``\nand ``openwisp2fw_organizations.categories`` need to be configured\nas in the example below:\n\n```yaml\n- hosts:\n    - myhost\n  roles:\n    - openwisp.openwisp2-imagegenerator\n  vars:\n    openwisp2fw_controller_url: \"https://openwisp.myproject.com\"\n    openwisp2fw_organizations:\n      - name: staging\n        flavours:\n          - default\n        openwisp:\n          url: \"{{ openwisp2fw_controller_url }}\"\n          shared_secret: \"xxxxx\"\n        root_password: \"xxxxx\"\n        categories:\n          default: \u003cCATEGORY-UUID\u003e\n      - name: prod\n        flavours:\n          - default\n        openwisp:\n          url: \"{{ openwisp2fw_controller_url }}\"\n          shared_secret: \"xxxxx\"\n        root_password: \"xxxxx\"\n        categories:\n          default: \u003cCATEGORY-UUID\u003e\n    openwisp2fw_uploader:\n        enabled: true\n        url: \"{{ openwisp2fw_controller_url }}\"\n        token: \"\u003cREST-API-USER-TOKEN\u003e\"\n        image_types:\n            - ath79-generic-ubnt_airrouter-squashfs-sysupgrade.bin\n            - ar71xx-generic-ubnt-bullet-m-xw-squashfs-sysupgrade.bin\n            - ar71xx-generic-ubnt-bullet-m-squashfs-sysupgrade.bin\n            - octeon-erlite-squashfs-sysupgrade.tar\n            - ath79-generic-ubnt_nanostation-loco-m-xw-squashfs-sysupgrade.bin\n            - ath79-generic-ubnt_nanostation-loco-m-squashfs-sysupgrade.bin\n            - ath79-generic-ubnt_nanostation-m-xw-squashfs-sysupgrade.bin\n            - ar71xx-generic-ubnt-nano-m-squashfs-sysupgrade.bin\n            - ath79-generic-ubnt_unifiac-mesh-squashfs-sysupgrade.bin\n            - x86-64-combined-squashfs.img.gz\n            - x86-generic-combined-squashfs.img.gz\n            - x86-geode-combined-squashfs.img.gz\n            - ar71xx-generic-xd3200-squashfs-sysupgrade.bin\n```\n\nThe following placeholders in the example will have to be substituted:\n\n- `\u003cCATEGORY-UUID\u003e` is the UUID o the firmware category in OpenWISP Firmware Upgrader\n- `\u003cREST-API-USER-TOKEN\u003e` is the REST auth token of a user with permissions to upload images\n\nYou can retrieve the REST auth token by sending a POST request using the Browsable API web interface of OpenWISP:\n\n1. Open the browser at `https://\u003copenwisp-base-url\u003e/api/v1/user/token/`.\n2. Enter username and password in the form at the bottom of the page.\n3. Submit the form and you will get the REST auth token in the response.\n\nThe upload script creates a new build object and then uploads the firmware images\nspecified in `image_types`, which have to correspond to the identifiers like\n`ar71xx-generic-tl-wdr4300-v1-il-squashfs-sysupgrade.bin` defined in the\n[hardware.py file of OpenWISP Firmware Upgrader](https://github.com/openwisp/openwisp-firmware-upgrader/blob/master/openwisp_firmware_upgrader/hardware.py).\n\nOther important points to know about the `upload_firmware.py` script:\n\n- The script reads `CONFIG_VERSION_DIST` and `CONFIG_VERSION_NUMBER`\n  from the `.config` file of the OpenWrt source code to determine the build\n  version.\n- The script will find out if a build with the same version and category already exists\n  and try to add images to that build instead of creating a new one, if duplicates\n  are found, a failure message will be printed to the console but the script will not\n  terminate; this allows to generate images for new hardware models and\n  add them to existing builds\n\nAdding files to images\n======================\n\nYou can add arbitrary files in every generated image by placing these files in a directory named\n`files/` in your playbook directory.\n\nExample:\n\n```\n.\n├── hosts\n├── playbook.yml\n└── files/etc/profile\n```\n\n`files/etc/profile` will be added in every generated image.\n\nAdding files for specific organizations\n=======================================\n\nYou can add files to images of specific organizations too.\n\nLet's say you have an organization called `snakeoil` and you want to add a custom banner,\nyou can accomplish this by creating the following directory structure:\n\n```\n.\n├── hosts\n├── playbook.yml\n└── organizations/snakeoil/etc/banner\n```\n\nSince this step is one of the last steps performed before building\nthe final images, you can use this feature to overwrite any file\nbuilt automatically during the previous steps.\n\nExtra parameters\n================\n\nYou can pass the following extra parameters to `ansible-playbook`:\n\n* `recompile`: wether to repeat the compilation step\n* `cores`: number of cores to use during the compilation step\n* `orgs`: comma separated list of organization names if you need to\n  limit the generation of images to specific organiations\n\n### Examples\n\nRecompile with 16 cores:\n\n```\nansible-playbook -i hosts playbook.yml -e \"recompile=1 cores=16\"\n```\n\nGenerate images only for organization ``foo``:\n\n```\nansible-playbook -i hosts playbook.yml -e \"orgs=foo\"\n```\n\nGenerate images only for organizations ``foo`` and ``bar``:\n\n```\nansible-playbook -i hosts playbook.yml -e \"orgs=foo,bar\"\n```\n\nRun specific steps\n==================\n\nSince each step in the process is tagged, you can run specific steps by using ansible tags.\n\nExample 1, run only the preparation of generators:\n\n```\nansible-playbook -i hosts playbook.yml -t generator\n```\n\nExample 2, run only the preparation of generators and build steps:\n\n```\nansible-playbook -i hosts playbook.yml -t generator,build\n```\n\nTargets with no subtarget\n=========================\n\nThis example shows how to fill `openwisp2fw_source_targets` in\norder to compile targets that do not specify a subtarget\n(eg: sunxi, ARMv8, QEMU):\n\n```yaml\nopenwisp2fw_source_targets:\n    # Allwinner SOC, Lamobo R1\n    - system: sunxi\n      profile: sun7i-a20-lamobo-r1\n    # QEMU ARM Virtual Image\n    - system: armvirt\n      profile: Default\n```\n\nSupport\n=======\n\nSee [OpenWISP Support Channels](http://openwisp.org/support.html).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenwisp%2Fansible-openwisp2-imagegenerator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenwisp%2Fansible-openwisp2-imagegenerator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenwisp%2Fansible-openwisp2-imagegenerator/lists"}