Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/bfgroup/ci_playground

Playground for Cloud CI development for C++
https://github.com/bfgroup/ci_playground

appveyor appveyor-ci azure-pipelines c-plus-plus ci circleci cirrus-ci clang cplusplus cpp gcc mingw mingw-w64 msvc semaphore-ci semaphoreci travis-ci visual-studio xcode

Last synced: 9 days ago
JSON representation

Playground for Cloud CI development for C++

Awesome Lists containing this project

README 

          
= Continuous Integration Playground



This is a "playground" for testing out basic methods to get Cloud CI systems

working with minimal effort. Not just getting any CI working, but specifically

getting C++ to work in CI.



== Goals



This project is of limited scope. In particular it follows these goals:



*C+\+*: Sufficient support to install and use a C\++ toolset to compile and

run C++ programs.



*Free*: We only consider services that are available to use for free to some

capacity. This usually means that the compute resources and features are

limited.



*Simple*: The configurations are as simple as possible to get specific toolsets

working and nothing more. This means there is no build system setup. And the

few small scripts used are there to minimize duplication.



*DRY*: The configurations try to avoid repeated definitions as much as is

possible within the restrictions of the particular service.



== General Configuration



All the included services are configured through configuration files included

in the source repository. Consult the individual service descriptions below

for what those files are.



=== Setup Scripts



In order to avoid duplication we use some minimal install and setup scripts

instead of including all the commands in the service configurations themselves.



link:.ci_playground/linux-cxx-install.sh[/.ci_playground/linux-cxx-install.sh]::

	Will install a set of packages indicated in the service configuration

	environment variable `PACKAGES`. That can be any set of packages, i.e.

	both compilers and support packages. Additionally it can set up the LLVM

	APT repository through `LLVM_OS` and `LLVM_VER` environment variables.



link:.ci_playground/macos-xcode-install.sh[/.ci_playground/macos-xcode-install.sh]::

	Finds the indicated Xcode, in the `XCODE_VERSION` environment variable,

	present in the `/Applications` directory and sets it as the active Xcode.

	Alternatively, uses `xcode-install` tool to install and/or setup Xcode as

	indicated in the `XCODE_VERSION` environment variable. See below for

	additional setup needed for the install to work.



link:.ci_playground/windows-msvc-setup.cmd[/.ci_playground/windows-msvc-setup.cmd]::

	Uses the link:https://github.com/Microsoft/vswhere[`vswhere`] utility to

	find the indicated, in the `VSWHERE_VERSION` or `VSWHERE_LEGACY`

	environment variables, and to run the `vcvarsall.bat` setup script. The

	`vcvarsall.bat` script from Visual Studio will set up the execution

	environment for calling `cl.exe` to compile directly.



=== macOS Xcode



Setting up Xcode is simply a matter of setting the `XCODE_VERSION` env variable

to the version to be tested. The setup script will search through the

pre-installed Xcode installations in the `/Applications` directory.



Optionally there's support for using the great work of the Fastlane developers

and the https://github.com/xcpretty/xcode-install[`xcode-install`] tool. It

makes setting up consistent Xcode installs easy. Additionally it makes it

possible to fresh install a full Xcode and command line tools.



Without any additional configuration we use `xcode-install` to select which

Xcode version to use. But, by setting up Apple Developer credentials we can use

it to download and install additional Xcode versions (within the limits of

macOS and Xcode compatibility constraints). The setup amounts to setting two

environment variables:



`XCODE_INSTALL_USER`:: User name for Apple Developer account.

`XCODE_INSTALL_PASSWORD`:: Password, or token, for the account.



Check below for how those variables are defined for specific services.



WARNING: The support for using `xcode-install` is no longer tested as Apple

has made it almost impossible to access their services within automation

programs.



== Services



The cloud Ci services vary in features from each other but generally they all

provide equivalent free services. One big difference though is what platforms

they include in their free offerings. Universally, Linux resources are freely

available. Some offer Windows resources to some extent. Additionally some

offer macOS resources, but usually this is behind a paid subscription. We also

only include services that support the GitHub revision control service.



Below we only discuss the resources available in the free tiers of the

services.



=== https://www.appveyor.com/[AppVeyor] image:https://ci.appveyor.com/api/projects/status/ig1acmghu4we5be2/branch/master?svg=true["Build Status", link="https://ci.appveyor.com/project/bfgroup/ci-playground"]



Configuration::

	link:appveyor.yml[`/appveyor.yml`]



AppVeyor provides CI resources for Linux, macOS, and Windows. For Windows they

provide a large array of pre-configured tools. For macOS, and hence Xcode, to

make use of the `xcode-install` tool you will need to create a

https://www.appveyor.com/docs/build-configuration/#secure-variables[secure value]

for both `XCODE_INSTALL_USER` and `XCODE_INSTALL_PASSWORD`. And place the

secure values in the `appveyor.yml` file.



=== https://azure.microsoft.com/en-us/services/devops/pipelines/[Azure Pipelines] image:https://dev.azure.com/bfgroup/CI%20Playground/_apis/build/status/bfgroup.ci_playground?branchName=master["Build Status", link="https://dev.azure.com/bfgroup/CI%20Playground/_build/latest?definitionId=4&branchName=master"]



Configuration::

	link:azure-pipelines.yml[`/azure-pipelines.yml`]



Microsoft Azure Pipelines offers Linux, macOS, and Windows resources. For using

`xcode-install` Azure Pipelines doesn't provide for a direct way of setting the

credential environment variables we need. Instead we need to define

https://docs.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#secret-variables[secret variables] that we use to set the

environment variables from. Define `DEV_APPLE_USER` and `DEV_APPLE_PASSWORD`

secret variables corresponding to `XCODE_INSTALL_USER` and `XCODE_INSTALL_PASSWORD`.



=== https://circleci.com/[CircleCI] image:https://circleci.com/gh/bfgroup/ci_playground/tree/master.svg?style=shield["Build Status", link="https://circleci.com/gh/bfgroup/ci_playground/tree/master"]



Configuration::

	link:.circleci/config.yml[`/.circleci/config.yml`]



CircleCI only gives us only Linux systems for free. Having the configuration

file is the only setup needed for CircleCI.



=== https://cirrus-ci.org/[Cirrus CI] image:https://api.cirrus-ci.com/github/bfgroup/ci_playground.svg?branch=master["Build Status", link="https://cirrus-ci.com/github/bfgroup/ci_playground"]



Of the current cloud CI offerings Cirrus CI is different in that it offers

FreeBSD public VMs. Cirrus CI provides "community clusters" that also include

Linux, macOS, and Windows. Unfortunately getting macOS Xcode and Visual Studio

installed is currently not possible.



Configuration::

	link:.cirrus.yml[`/.cirrus.yml`]



=== https://drone.io/[Drone] image:https://cloud.drone.io/api/badges/bfgroup/ci_playground/status.svg?ref=refs/heads/master["Build Status", link="https://cloud.drone.io/bfgroup/ci_playground"]



Configuration::

	link:.drone.star[`/.drone.star`]



Free cloud access is available for Linux on Drone. There are a few ways to go

with the configuration in Drone. The main configuration, like others, is done

through a yaml file (`/.drone.yml`). But for our use case, where we want

to do the same thing for many different toolsets, it means lots of duplication

as it has no built-in support for DRY. Drone does support alternate,

programmatic, methods for specifying the configuration with either

link:https://docs.drone.io/pipeline/scripting/jsonnet/[Jsonnet] or

link:https://docs.drone.io/pipeline/scripting/starlark/[Starlark] scripting.

For this we use a Starlark script to create all the toolset variations. To

make it work one needs to point Drone to the alternate configuration script

in the project settings.



=== https://help.github.com/en/actions[GitHub Actions] image:https://github.com/bfgroup/ci_playground/workflows/C++%20Tooling/badge.svg?branch=master&event=push["Build Status", link="https://github.com/bfgroup/ci_playground/actions"]



Configuration::

	link:.github/workflows/cxx_tooling.yml[`/.github/workflows/cxx_tooling.yml`]



GitHub Actions provides Linux, macOS, and Windows host resources to test with.

There isn't any setup past creating at least one "workflow" configuration file.

Multiple workflows are supported if you want to segregate your builds. For our

case we only need the one configuration file. To use `xcode-install` GitHub

Actions provides a way to define account global link:https://help.github.com/en/actions/configuring-and-managing-workflows/creating-and-storing-encrypted-secrets[secrets]

that can be extracted in the YAML configuration. In our case we define

`dev_apple_user` and `dev_apple_password` secret variables

corresponding to `XCODE_INSTALL_USER` and `XCODE_INSTALL_PASSWORD` in your

account.



=== https://semaphoreci.com/[Semaphore] image:https://bfgroup.semaphoreci.com/badges/ci_playground/branches/master.svg?style=shields["Build Status", link="https://bfgroup.semaphoreci.com/branches/9a7e94b0-124b-47b8-a0e3-3d633533753a"]



Configuration::

	link:.semaphore/semaphore.yml[`/.semaphore/semaphore.yml`]



Semaphore gives us Linux and macOS free build capabilities. The configuration

specification is a bit different than other CI systems in some ways though.

It doesn't support any built-in way to reuse setups (i.e. to stay DRY). And

the usual YAML method of using the `<<` map merge key is not allowed as the

configuration is strictly checked against a schema. For our case we create

some dummy (i.e. skipped) blocks that we can use to apply the `*name` YAML

expansion with. Which reduces much of the repetition. To use `xcode-install`

one can define global account "Secrets" in the web UI for encrypted

environment variables.

Define `DEV_APPLE_USER` and `DEV_APPLE_PASSWORD` encrypted variables

corresponding to `XCODE_INSTALL_USER` and `XCODE_INSTALL_PASSWORD` in your

account as a `dev-apple` secret.



=== https://travis-ci.com/[Travis CI] image:https://travis-ci.com/bfgroup/ci_playground.svg?branch=master["Build Status", link="https://travis-ci.com/bfgroup/ci_playground"]



Configuration::

	link:.travis.yml[`/.travis.yml`]



Travis CI provides Linux, macOS, and experimental Windows support. Although

we don't have Windows as part of our configuration yet (soon). The Travis CI

configuration is perhaps the simples of all the CI systems. As one can

precisely define each variation to build individually. For using

`xcode-install` Travis CI allows one to set encrypted environment variables.

Define `DEV_APPLE_USER` and `DEV_APPLE_PASSWORD` encrypted variables

corresponding to `XCODE_INSTALL_USER` and `XCODE_INSTALL_PASSWORD` in your

project.



== Copyright



This work is Copyright René Ferdinand Rivera Morell 2020-2021. And distributed, and subject to,

the link:LICENSE.txt[Boost Software License, Version 1.0].