{"id":31953006,"url":"https://github.com/dkogan/base-os-builder","last_synced_at":"2026-02-26T00:50:46.914Z","repository":{"id":311079141,"uuid":"1042352295","full_name":"dkogan/base-os-builder","owner":"dkogan","description":null,"archived":false,"fork":false,"pushed_at":"2025-08-22T00:26:58.000Z","size":9,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-08-22T02:33:08.304Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Makefile","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dkogan.png","metadata":{"files":{"readme":"README.org","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-21T22:05:50.000Z","updated_at":"2025-08-22T00:27:01.000Z","dependencies_parsed_at":"2025-08-22T02:33:25.839Z","dependency_job_id":"b998fa82-a50a-4fab-bc8a-4d1ae996b1ea","html_url":"https://github.com/dkogan/base-os-builder","commit_stats":null,"previous_names":["dkogan/base-os-builder"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/dkogan/base-os-builder","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dkogan%2Fbase-os-builder","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dkogan%2Fbase-os-builder/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dkogan%2Fbase-os-builder/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dkogan%2Fbase-os-builder/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dkogan","download_url":"https://codeload.github.com/dkogan/base-os-builder/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dkogan%2Fbase-os-builder/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279018711,"owners_count":26086609,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-14T02:00:06.444Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-14T13:27:49.684Z","updated_at":"2025-10-14T13:27:52.636Z","avatar_url":"https://github.com/dkogan.png","language":"Makefile","funding_links":[],"categories":[],"sub_categories":[],"readme":"* Overview\n\nBuild dependency meta-packages and OS tarballs that use those packages\n\n* Synopsis\n\n#+begin_example\n$ FLAVORS='native cross-arm64 robot-arm64'                                \\\n  DEBIAN_RELEASE=trixie                                                   \\\n  PROJECT=sample-project                                                  \\\n  APT_REPOS_EXTRA=https://fatty.secretsauce.net/debian/sample-project/    \\\n  DPUT_TARGET=sample-project-fatty-trixie                                 \\\n  APTOPT='Acquire::https::fatty.secretsauce.net::Verify-Peer=false'       \\\n  make deps                                                               \\\n#+end_example\n\nWe built the 3 dependency meta-packages for the 3 flavors we asked for: a\nnative-amd64 one, a native-arm64 one and a cross-building amd64-arm64 one. The\ndependent packages are defined in the [[file:sample-project-deps.equivs][=sample-project-deps.equivs= file]]. The\nrelevant data in each meta-package\n\n#+begin_example\n$ dpkg -I sample-project-deps-native_0.01_amd64.deb\n  Package: sample-project-deps-native\n  Architecture: amd64\n  Depends: build-essential, mrcal, libmrcal-dev, devscripts, python3\n  ...\n\n$ dpkg -I sample-project-deps-cross-arm64_0.01_amd64.deb\n  Package: sample-project-deps-cross-arm64\n  Architecture: amd64\n  Depends: crossbuild-essential-arm64, libmrcal-dev:arm64, gdb, python3\n  ...\n\n$ dpkg -I sample-project-deps-robot-arm64_0.01_arm64.deb\n  Package: sample-project-deps-robot-arm64\n  Architecture: arm64\n  Depends: build-essential, mrcal, libmrcal-dev, devscripts, python3\n  ...\n#+end_example\n\nI push those meta-packages to the local APT repository that everybody uses to\nget custom packages for this project:\n\n#+begin_example\n$ FLAVORS='native cross-arm64 robot-arm64'                                \\\n  DEBIAN_RELEASE=trixie                                                   \\\n  PROJECT=sample-project                                                  \\\n  APT_REPOS_EXTRA=https://fatty.secretsauce.net/debian/sample-project/    \\\n  DPUT_TARGET=sample-project-fatty-trixie                                 \\\n  APTOPT='Acquire::https::fatty.secretsauce.net::Verify-Peer=false'       \\\n  make push-deps\n#+end_example\n\nIn addition to pushing to the APT server, this makes a git tag, and pushes that.\nAny machines already set up using these packages can now move to the latest ones\nwith the usual\n\n#+begin_src sh\napt upgrade \u0026\u0026 apt update\n#+end_src\n\nBut to make it easier to set up brand-new machines, we can also make OS tarballs\nthat can be extracted and used directly. This is usually done in a chroot\nmanaged by =schroot=, but any similar tooling can work.\n\n#+begin_example\n$ FLAVORS='native cross-arm64 robot-arm64'                                \\\n  DEBIAN_RELEASE=trixie                                                   \\\n  PROJECT=sample-project                                                  \\\n  APT_REPOS_EXTRA=https://fatty.secretsauce.net/debian/sample-project/    \\\n  DPUT_TARGET=sample-project-fatty-trixie                                 \\\n  APTOPT='Acquire::https::fatty.secretsauce.net::Verify-Peer=false'       \\\n  make tarballs\n#+end_example\n\nWe can also select specific tarballs instead of /all/ of them with something\nlike\n\n#+begin_src sh\n.... make tarballs-native\n#+end_src\n\nAs usual, the available targets can be listeed with\n\n#+begin_src sh\n.... remake --list-targets\n#+end_src\n\n* Description\n\nThis toolkit solves a common problem faced by big-ish software projects: how to\nmanage the software configuration on /all/ the computers. There are many efforts\nto do this. The scenario targeted by /this/ project:\n\n- Different people might be using machines with different architectures. I\n  routinely encounter groups with mixed amd64/arm64 boxes used for development,\n  and everyone should be supported, while making sure that the versions of the\n  libraries /everybody/ is using are identical\n\n- Different machines might have different specific purposes. I routinely\n  encounter projects that build software for a target machine (a robot) equipped\n  with a slower arm64 processor, while everybody's workstations use much faster\n  CPUs and a mix of architectures. The robot doesn't need to have all the\n  development tooling. But the libraries it does have /must/ be exactly those on\n  the dev machines\n\n- The versions of everything must match, so if a test passes on somebody's dev\n  machine, we should have a reasonable expectation that it will work on the\n  robot.\n\n- Cross-building must work: building for the robot should be trivial, even if\n  the dev machine is using a different architecture\n\n- Setting everything up and managing things and updating things must be trivial.\n  A page-long human script that pulls stuff from Debian and pip and conda and\n  some sources from git, which it then makes available with =sudo make install=\n  isn't acceptable.\n\nThe approach we take leans heavily on Debian:\n\n- Everybody agrees on a base OS. The latest release of Debian (trixie at the\n  time of this writing) is /strongly/ recommended. This supports /lots/ of\n  architecture and has /lots/ of libraries and tools available to do almost\n  anything\n\n- People running a different Linux distro can install the common OS into a\n  subdirectory on disk and =chroot= into it. =schroot= strongly recommended to\n  do this\n\n- /All/ dependencies come from packages. Anything that isn't available in stock\n  Debian is packaged and pushed to a local APT server\n\n- /All/ dependencies required for a particular task are made available in a\n  metapackage. So a fresh install of the OS + this metapackage produces a system\n  that can do everything required for this project. Different tasks might\n  require different meta-packages; in the example above we build 3 flavors:\n\n  - =sample-project-deps-native= for a native amd64 dev machine\n  - =sample-project-deps-cross-arm64= for a machine meant to cross build\n    amd64-\u003earm64\n  - =sample-project-deps-robot-arm64= for a native arm64 machine meant for\n    deployment (not development)\n\nThe tooling in this package is accessed through the [[file:Makefile][=Makefile=]]. You can =make\ndeps= to build the meta-packages, =make push-deps= to push them to the APT\nserver and =make tarballs= to build OS tarballs with the meta-package\npreinstalled (see the sample invocation above).\n\nMetapackage definitions are defined in a file filtered by the =Makefile= and\nthen processed by the =equivs= tool to create the package. The OS tarballs are\nmade with =mmdebstrap=.\n\nNew installs can be done by installing a fresh OS and then installing the\nmetapackage. /Or/ the pre-cannedtarball can be extracted into a fresh directory.\n\nAlready-installed boxes can be updated with the usual =apt update \u0026\u0026 apt\nupgrade=.\n\n* User-facing documentation\n** Installing the system\n*** On an existing install of the common distro\nIf you're already running the common Linux distribution, there's no need for any\nchild OS, and the metapackage can be used directly.\n\nYou should add the custom project-specific APT repository to your\n=/etc/apt/sources.list=; something like:\n\n#+begin_example\ndeb [trusted=yes] https://fatty.secretsauce.net/debian/sample-project/ trixie main\n#+end_example\n\nThen\n\n#+begin_src sh\nsudo apt update \u0026\u0026 \\\nsudo apt install sample-project-deps-native\n#+end_src\n\nThat's it. If cross-building, a bit more setup is needed and the\ncross-building-specific metapackage is needed. Something like this:\n\n\n#+begin_src sh\ndpkg --add-architecture arm64\n\nsudo apt update \u0026\u0026 \\\nsudo apt install sample-project-deps-cross-arm64\n#+end_src\n\nNote that a number of Debian packages still aren't co-installable with multiple\narchitectures at the same time. So for cross-building, it might still be\nrequired to make a child OS, as noted in the following section\n\n*** Setting up schroot\nIf you're running a different Linux distro, the =schroot= tool is recommended as\na shell to use libraries from a child OS. Assuming a Debian-style host OS:\n\n#+begin_src sh\nsudo apt install schroot\n#+end_src\n\nThen drop something like this into =/etc/schroot/chroot.d/sample-project=:\n\n#+begin_src sh\n[sample]\ndescription=Debian/trixie\nusers=USERNAME\nroot-users=USERNAME\ngroups=sbuild\nroot-groups=root\ntype=directory\ndirectory=DIRECTORY\nshell=/bin/bash\npreserve-environment=yes\nprofile=desktop\n#+end_src\n\nadjusting the =USERNAME= and =DIRECTORY= to your username and the directory\nwhere the common OS will be installed respectively. The OS can be \"installed\" by\nextracting the convenience tarballs created by this toolkit:\n\n#+begin_src sh\ncd DIRECTORY\ntar xvfz /tmp/sample-project-native_0.01_amd64.tar.gz\n#+end_src\n\nDepending on various uninteresting things, this may produce benign, ignorable\nwarnings about =mknod= or =/dev=.\n\nThat's it. The chroot is now installed and ready to use. All dependencies\nrequired to build and run stuff are available.\n\n*** On a non-Linux OS\nSome people like using non-linux OSs with some sort of virtualization layer to\nget a Linux machine. In this case it is strongly recommended to skip the\nschroot, and to install the common distribution directly into the virtual\nenvironment, and then set stuff up using the above notes.\n\n** Using the installed system\n*** Where should I put my code when working inside the chroot?\nThis came up several times. The chroot provides its own system paths (=/usr=,\n=/lib=, =/run=, ....) but your own stuff is shared. In particular, =$HOME= is\nthe same inside and outside the chroot, and the data can be organized in\nwhatever way you like.\n\n*** How do I do anything in the chroot?\nYou do everything exactly how you would normally, except you drop into the\nchroot prior to running the command. So you do one of\n\n- run =schroot -c sample= to get an interactive shell, and then run commands\n  inside\n- Prefix your commands with =schroot -c sample --=. So if you want to run =cmd a\n  b c=, say =schroot -c sample -- cmd a b c=\n\n*** Updating the chroot\n#+begin_src sh\nsudo schroot -c bookworm -- 'apt update \u0026\u0026 apt upgrade'\n#+end_src\n\n*** Deleting the chroot\nWARNING: Your =/home= directory is bind mounted within the chroot, so all of the\nsame files and repos are accessible. This also means it can be dangerous to\ndelete the chroot directory, DIRECTORY before unmounting. Here are some tips:\n\n- As long as there are no active schroot sessions, there should be no bind\n  mounts.\n- You can delete all active sessions: =schroot --all-sessions --end-session=\n- To be extra paranoid, running =mount | grep sample= should return nothing\n- To be extra extra paranoid, running =ls DIRECTORY/bookworm/home= should return\n  nothing\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdkogan%2Fbase-os-builder","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdkogan%2Fbase-os-builder","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdkogan%2Fbase-os-builder/lists"}