{"id":14988041,"url":"https://github.com/apache/couchdb-ci","last_synced_at":"2025-10-28T16:41:26.667Z","repository":{"id":34578634,"uuid":"38525117","full_name":"apache/couchdb-ci","owner":"apache","description":"Apache CouchDB Continuous Integration (CI) support repository","archived":false,"fork":false,"pushed_at":"2024-12-10T02:20:36.000Z","size":10559,"stargazers_count":10,"open_issues_count":2,"forks_count":16,"subscribers_count":19,"default_branch":"main","last_synced_at":"2025-01-30T10:34:30.255Z","etag":null,"topics":["apache","big-data","cloud","couchdb","cplusplus","database","erlang","http","javascript","network-client","network-server"],"latest_commit_sha":null,"homepage":"https://github.com/apache/couchdb-ci","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/apache.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2015-07-04T07:00:05.000Z","updated_at":"2024-12-10T02:20:37.000Z","dependencies_parsed_at":"2024-01-02T14:57:03.825Z","dependency_job_id":"157a202b-1f0c-451b-8a56-5df33a9f46e9","html_url":"https://github.com/apache/couchdb-ci","commit_stats":{"total_commits":218,"total_committers":21,"mean_commits":"10.380952380952381","dds":0.6330275229357798,"last_synced_commit":"50564bea320a696a0a4385cb7b47e243d594a693"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Fcouchdb-ci","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Fcouchdb-ci/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Fcouchdb-ci/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/apache%2Fcouchdb-ci/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/apache","download_url":"https://codeload.github.com/apache/couchdb-ci/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":237125530,"owners_count":19259300,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache","big-data","cloud","couchdb","cplusplus","database","erlang","http","javascript","network-client","network-server"],"created_at":"2024-09-24T14:15:59.736Z","updated_at":"2025-10-19T12:30:34.156Z","avatar_url":"https://github.com/apache.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CouchDB Continuous Integration (CI) support repo\n\nThe main purpose of this repository is to provide scripts that:\n\n* Install the necessary build-time dependencies for CouchDB on a number of platforms, either inside or outside of a container or VM\n* Build Docker containers with those dependencies necessary to build binary JavaScript (SpiderMonkey 1.8.5) packages\n* Build Docker containers with all dependencies necessary to build CouchDB, including Erlang and JavaScript\n\nIt intends to cover a range of both operating systems (Linux, macOS, BSD, Windows) and Erlang versions (17.x, 18.x, 19.x, etc.)\n\nThese images are used by [Apache Jenkins CI](https://ci-couchdb.apache.org/blue/organizations/jenkins/pipelines) to build CouchDB with every checkin to `main`, `3.x`, a release branch (*e.g.*, `2.3.0`), or an open Pull Request. CouchDB's CI build philosophy is to validate CouchDB against different Erlang versions with each commit to a Pull Request, and to validate CouchDB against different OSes and architectures on merged commits to `main`, `3.x`, and release branches. Where possible, Jenkins also auto-builds convenience binaries or packages. The eventual goal is that these auto-built binaries/packages/Docker images will be auto-pushed to our distribution repos for downstream consumption.\n\n# Supported Configurations\n\nSee Docker Hub for the latest supported images:\n\n- https://hub.docker.com/r/apache/couchdbci-debian/tags\n- https://hub.docker.com/r/apache/couchdbci-ubuntu/tags\n- https://hub.docker.com/r/apache/couchdbci-centos/tags\n\n---\n\n# Docker\n\nFor those OSes that support Docker, we run builds inside of Docker containers. These containers are built using the `build.sh` command at the root level.\n\n## Authenticating to Docker Hub\n\n1.  You need a Docker Cloud account with access to the `apache` organization to upload images. Ask the CouchDB PMC for assistance with this.\n2. `export DOCKER_ID_USER=\"username\"`\n3. `docker login -u $username` and enter your password. (If using `podman` specify the registry `docker login -u $username docker.io`)\n\n## Building a \"platform image\"\n\nThe platform images include all of the build dependencies necessary to build and full test CouchDB on a given OS/version/architecture combination.\n\nBuild a platform image with:\n\n```\n./build.sh platform \u003cdistro\u003e-\u003cversion\u003e\n```\n\n## Overriding the Erlang, Elixir or Node version\n\nWe want to generate a `rebar` binary compatible with all versions of Erlang we support. If we do this on too new a version, older Erlangs won't recognize it. So we always keep an image around with that version.\n\nOn the other hand, some OSes won't run older Erlangs because of library changes, so you need to override that environment variable.\n\nJust specify on the command line any of the `ERLANGVERSION`, `NODEVERSION`, or `ELIXIRVERSION` environment variables:\n\n```\nNODEVERSION=20 ELIXIRVERSION=v1.18.4 ERLANGVERSION=26.2.5.13 ./build.sh platform debian-bookworm\n```\n\n## Building images for other architectures\n\n### Multi-arch images with Docker Buildx\n\nUse Docker's\n[Buildx](https://docs.docker.com/buildx/working-with-buildx/) plugin to generate\nmulti-architecture container images with a single command invocation.\n\nMulti-arch images can be built locally, with archtectures emulated\nwith qemu, or using remote workers, if we have ssh acccess to hosts\nrunning on each of the supported architectures. Aside from the initial\nsetup the rest of the commands are exactly the same.\n\n##### Local setup\n\n```\ndocker buildx create --name apache-couchdb --use\n```\n\n##### Remote setup\n\nThis requires your ssh key to be installed on the remote workers. See\nthe `couchdb-infra-cm` repo for the list of current keys.\n\nTo use shorter names for the servers can include the `ssh.cfg` file\nfrom `couchdb-infra-cm` in your `ssh_config`.\n\nEnsure ssh connection to each build server works. There maybe be a\ndifferent user for each one (root, ubuntu, linux1, etc), the `ssh.cfg`\nfile, if you're using that already handles that transparently.\n\nIf the user is not root, ensure the user can run `docker ps`. For instance:\n```\nsudo usermod -aG docker linux1\n```\n\nTest that `docker ps` run for each server:\n\n```\nssh linux1 docker ps\nssh ubuntu-nc-arm64-12 docker ps\nssh ubuntu docker ps\nssh ubuntu-fra1-10 docker ps\n```\n\nSetup the `multiarch` xbuild context:\n\n```\ndocker buildx rm multiarch  || true\ndocker buildx create --name multiarch --driver docker-container --platform linux/arm64 ssh://ubuntu-nc-arm64-12\ndocker buildx create --append --name multiarch --driver docker-container --platform linux/amd64 ssh://ubuntu-fra1-10\ndocker buildx create --append --name multiarch --driver docker-container --platform linux/s390x ssh://linux1\ndocker buildx create --append --name multiarch --driver docker-container --platform linux/ppc64le ssh://ubuntu\n```\n\nBefore building, set `docker buildx` to `use` the new context as the new default.\nWhen done, set to `default`, or whatever it was before:\n\n```\ndocker buildx use multiarch\n```\n\n#### Building\n\nThe `build.sh` script has `buildx-base` and `buildx-platform` targets that will\nwill build **and upload** a new multi-arch container image to the registry. For\nexample:\n\n```\n./build.sh buildx-platform debian-bookworm\n```\n\nThe `$BUILDX_PLATFORMS` environment variable can be used to override the default\nset of target platforms that will be supplied to the buildx builder.\n\n# Useful things you can do\n\n## Update images used for package releases with new Erlang versions\n\n```\nERLANGVERSION=27.3.3./build.sh buildx-platform-release\n```\n\nThis will build all the Debian and RHEL-clone OS images on x86-64 with that version of Erlang\n\n## Update images used for CI with new Erlang versions\n\n```\nERLANGVERSION=24.3.4.7 ./build.sh buildx-platform debian-bookworm\n```\n\nThis will update Debian Bullseye OS image for all architectures (x86,\nppc64le, arm64) with that version of Erlang. Do this for the images\nwhich are used with non-x86 architectures.\n\nIf the same image is used for release package building and CI, run\nthis command after `buildx-platform-release` to ensure that the\ndebian-bullseye will be a rebuilt as a multi-arch image. Otherwise,\nbuildx-platform-release creates as x86 only.\n\n## Update Debian Bullseye image with 25.2\n\n```\nBUILDX_PLATFORMS=linux/amd64 ERLANGVERSION=25.2 ./build.sh buildx-platform debian-bookworm\n```\n\nIn this case, since we're not using 25.2 for multi-arch testing, opt to build it only for x86.\n\n## Full `build.sh` options\n\n```\n./build.sh \u003ccommand\u003e [OPTIONS]\n\nRecognized commands:\n  clean \u003cplat\u003e              Removes all images for \u003cplat\u003e.\n  clean-all                 Removes all images for all platforms.\n\n  *buildx-platform \u003cplat\u003e   Builds a multi-architecture image with Erlang \u0026 JS support.\n  *buildx-platform-release \u003cplat\u003e Builds x86-64 images with default Erlang for all supported release OSes\n\n  couch \u003cplat\u003e              Builds and tests CouchDB for \u003cplat\u003e.\n  couch-all                 Builds and tests CouchDB on all platforms.\n\n  Commands marked with * require appropriate Docker Hub credentials.\n```\n\n## Interactively working in a built container\n\nAfter building the image as above:\n\n```\ndocker run -it couchdbdev/\u003ctag\u003e\n```\n\nwhere `\u003ctag\u003e` is of the format `\u003cdistro\u003e-\u003cversion\u003e-\u003ctype\u003e`, such as `debian-bookworm-erlang-26.2.5.11`.\n\n## Running the CouchDB build in a published container\n\n```\n./build.sh couch \u003cdistro\u003e-\u003cversion\u003e\n```\n\n## Building SpiderMonkey 1.8.5 convenience packages\n\nThis is only needed if a platform does not have a supported SpiderMonkey library. As of April 2021, this is no currently supported platform.\n\nFirst, build the 'base' image with:\n\n```\n./build.sh base \u003cdistro\u003e-\u003cversion\u003e\n```\n\nAfter building the base image as above, head over to the [apache/couchdb-pkg](https://github.com/apache/couchdb-pkg) repository and follow the instructions there.\n\n## Adding support for a new release/platform/architecture\n\n1. Update the build scripts in the `bin/` directory to install the dependencies correctly on your new OS/version/platform. Push a PR with these changes.\n1. Copy and customize an appropriate Dockerfile in the `dockerfiles` directory for your new OS.\n1. If a supported SpiderMonkey library is not available on the target platform, build a base image using `./build.sh base \u003cdistro\u003e-\u003cversion\u003e`. Solve any problems with the build process here.\n1. Using the [apache/couchdb-pkg](https://github.com/apache/couchdb-pkg) repository, validate you can build the JS package. Fix any problems in that repo that arise and raise a new PR. Open a new issue on that PR requesting that the JS packages be made available through the CouchDB repository/download infrastructure.\n1. Build a full platform image with `./build.sh platform \u003cdistro\u003e-\u003cversion\u003e`. Solve any problems with the build process here.\n1. Submit a PR against the [apache/couchdb](https://github.com/apache/couchdb) repository, adding the new platform to the top level `Jenkinsfile`. Ask if you need help.\n\n---\n\n# Other platforms\n\nWe are eager for contributions to enhance the build scripts to support setting up machines with the necessary build environment for:\n\n* NetBSD\n* OpenBSD\n* macOS\n* Windows x64 (see [apache/couchdb-glazier](https://github.com/apache/couchdb-glazier]) for the current approach)\n\nas well as alternative architectures for the already supported image types (armhf, ppc64le, s390x, sparc, etc).\n\nWe know that Docker won't support some of these, but we should be able to at least expand the install scripts for all of these platforms.\n\n# Background \n\nSee: \n* this [thread](https://www.mail-archive.com/dev%40couchdb.apache.org/msg43591.html) on the couchdb-dev mailing list and\n* this [ASF Infra ticket](https://issues.apache.org/jira/browse/INFRA-10126).\nfor the origins of this work.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapache%2Fcouchdb-ci","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fapache%2Fcouchdb-ci","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fapache%2Fcouchdb-ci/lists"}