https://github.com/cloudfoundry/buildpacks-ci
Concourse CI pipelines for the buildpacks team
https://github.com/cloudfoundry/buildpacks-ci
Last synced: 18 days ago
JSON representation
Concourse CI pipelines for the buildpacks team
- Host: GitHub
- URL: https://github.com/cloudfoundry/buildpacks-ci
- Owner: cloudfoundry
- License: apache-2.0
- Created: 2015-04-24T21:39:10.000Z (almost 11 years ago)
- Default Branch: master
- Last Pushed: 2026-01-22T15:44:17.000Z (21 days ago)
- Last Synced: 2026-01-23T07:27:50.099Z (20 days ago)
- Language: HTML
- Homepage: https://buildpacks.ci.cf-app.com
- Size: 15.5 MB
- Stars: 41
- Watchers: 36
- Forks: 41
- Open Issues: 15
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Notice: NOTICE
Awesome Lists containing this project
README
# Introduction
This contains the configuration for the Cloud Foundry Buildpacks team [Concourse deployment](https://buildpacks.ci.cf-app.com/).
# Pipelines
All pipelines are now generated using [ytt](https://carvel.dev/ytt/) templating.
* [dependency-builds](pipelines/dependency-builds): build binaries for Cloud Foundry buildpacks
* [buildpacks](pipelines/buildpack): test and release all of the buildpacks
* [brats](pipelines/brats): run [BRATS](https://github.com/cloudfoundry/brats) against the master branch of buildpacks
* [buildpack-verification](pipelines/buildpack-verification): generate static site for buildpack verification
* [cflinuxfs4](pipelines/cflinuxfs4.yml): cflinuxfs4 rootfs pipeline
# Concourse State
Jobs and tasks in the `buildpacks-ci` repository store state in [public-buildpacks-ci-robots](https://github.com/cloudfoundry/public-buildpacks-ci-robots). See repository README for details.
# Commands and recipes
## Updating all the Pipelines
```sh
./bin/update-pipelines
```
## Debugging the build
```sh
fly intercept -j $JOB_NAME -t task -n $TASK_NAME
```
## Clearing the git resources
```sh
fly intercept -c $RESOURCE_NAME rm -rf /tmp/git-resource-repo-cache
```
## To build a new version of a binary
1. Check out the `binary-builds` branch
2. Edit the YAML file appropriate for the build (e.g. `ruby-builds.yml`)
3. Find the version number and package SHA256 of the new binary. For many binaries, the project website provides the SHA256 along with the release (for example, jruby.org/download provides the SHA256 along with each JRuby release). For others (such as Godep), you download the .tar.gz file and run `shasum -a 256 ` to obtain the SHA256.
4. Add any number of versions and their checksums to the array, e.g.
```yaml
ruby:
- version: 2.2.2
sha256: 5ffc0f317e429e6b29d4a98ac521c3ce65481bfd22a8cf845fa02a7b113d9b44
```
5. `git commit -am 'Build ruby 2.2.2' && git push`
Build should automatically kick off at
https://buildpacks.ci.cf-app.com/pipelines/binary-builder and silently
upload a binary to the `buildpacks.cloudfoundry.org` bucket under
`dependencies/`,
e.g. https://buildpacks.cloudfoundry.org.s3.amazonaws.com/dependencies/ruby/ruby-2.2.2-linux-x64.tgz
Note that the array is a stack, which will be emptied as the build
succeeds in packaging successive versions.
## Making Changes to Build Scripts
When you want to change how a binary gets built, there are two places you may need to make changes. All binaries are built by the `dependency-builds` pipeline, and you may need to change the task that builds them. For many binaries, the `dependency-builds` pipeline runs recipes from the `binary-builder` repo; for those binaries, you will usually need to change the recipe rather than the concourse task.
For the list of currently supported binaries, check out our `dependency-builds` [pipeline](https://buildpacks.ci.cf-app.com/teams/main/pipelines/dependency-builds).
The concourse task that orchestrates the building is `buildpacks-ci/tasks/build-binary-new/builder.rb`; many of the recipes are in [binary-builder](https://github.com/cloudfoundry/binary-builder).
To test these changes locally, you can execute the concourse task for it, but point to local changes. For instance:
```
$ cd buildpacks-ci
$ STACK=cflinuxfs2 fly -t buildpacks e -c tasks/build-binary-new/build.yml -j dependency-builds/build-r-3.4.X -i buildpacks-ci=.
```
For binaries that use recipes in `binary-builder`, you can also test in Docker. For instance:
```
$ docker run -w /binary-builder -v `pwd`:/binary-builder -it cloudfoundry/cflinuxfs2:ruby-2.2.4 ./bin/binary-builder --name=ruby --version=2.2.3 --md5=150a5efc5f5d8a8011f30aa2594a7654
$ ls
ruby-2.2.3-linux-x64.tgz
```
# Buildpack Repositories Guide
`buildpacks-ci` pipelines and tasks refer to many other repositories. These repos are where the buildpack team and others develop buildpacks and related artifacts.
## Officially-supported Buildpacks
Each officially-supported buildpack has a `develop` and a `master` branch.
Active development happens on `develop`. Despite our best efforts, `develop` will sometimes be unstable and is not production-ready.
Our release branch is `master`. This is stable and only updated with new buildpack releases.
* [binary-buildpack](https://github.com/cloudfoundry/binary-buildpack)
* [go-buildpack](https://github.com/cloudfoundry/go-buildpack)
* [nodejs-buildpack](https://github.com/cloudfoundry/nodejs-buildpack)
* [php-buildpack](https://github.com/cloudfoundry/php-buildpack)
* [python-buildpack](https://github.com/cloudfoundry/python-buildpack)
* [ruby-buildpack](https://github.com/cloudfoundry/ruby-buildpack)
* [dotnet-core-buildpack](https://github.com/cloudfoundry/dotnet-core-buildpack)
* [staticfile-buildpack](https://github.com/cloudfoundry/static-buildpack)
## Tooling for Development and Runtime
* [buildpack-packager](https://github.com/cloudfoundry/buildpack-packager) Builds cached and uncached buildpacks
* [machete](https://github.com/cloudfoundry/machete) Buildpack integration testing framework.
* [compile-extensions](https://github.com/cloudfoundry/compile-extensions) Suite of utility scripts used in buildpacks at runtime
* [libbuildpack](https://github.com/cloudfoundry/libbuildpack) Library used for writing buildpacks in Golang
* [binary-builder](https://github.com/cloudfoundry/binary-builder) Builds binaries against specified rootfs
* [brats](https://github.com/cloudfoundry/brats) Buildpack Runtime Acceptance Test Suite, a collection of smoke tests
## BOSH Releases
BOSH releases are used in the assembly of [`cf-release`](https://github.com/cloudfoundry/cf-release).
* [go-buildpack-release](https://github.com/cloudfoundry/go-buildpack-release)
* [ruby-buildpack-release](https://github.com/cloudfoundry/ruby-buildpack-release)
* [python-buildpack-release](https://github.com/cloudfoundry/python-buildpack-release)
* [php-buildpack-release](https://github.com/cloudfoundry/php-buildpack-release)
* [nodejs-buildpack-release](https://github.com/cloudfoundry/nodejs-buildpack-release)
* [staticfile-buildpack-release](https://github.com/cloudfoundry/staticfile-buildpack-release)
* [binary-buildpack-release](https://github.com/cloudfoundry/binary-buildpack-release)
* [java-offline-buildpack-release](https://github.com/cloudfoundry/java-offline-buildpack-release)
* [java-buildpack-release](https://github.com/cloudfoundry/java-buildpack-release)
* [dotnet-core-buildpack-release](https://github.com/cloudfoundry/dotnet-core-buildpack-release)
## Experimental or unsupported
### Buildpacks
These buildpacks are possible candidates for promotion, or experimental architecture explorations.
* [hwc-buildpack](https://github.com/cloudfoundry/hwc-buildpack)
* [hwc-buildpack-release](https://github.com/cloudfoundry/hwc-buildpack-release)
### Tools
* [concourse-filter](https://github.com/pivotal-cf-experimental/concourse-filter) Redacts credentials from Concourse logs
* [new_version_resource](https://github.com/pivotal-cf-experimental/new_version_resource) Concourse resource to track dependency versions by scraping webpages
## Private Repos
Some repositories are private for historical or security reasons. We list them for completeness.
* [deployments-buildpacks](https://github.com/pivotal-cf/deployments-buildpacks) See repository README.
* [buildpacks-ci-robots](https://github.com/pivotal-cf/buildpacks-ci-robots) See repository README.