{"id":18419748,"url":"https://github.com/teradata/docker-images","last_synced_at":"2025-04-07T13:31:42.684Z","repository":{"id":146377845,"uuid":"69667280","full_name":"Teradata/docker-images","owner":"Teradata","description":"Docker images used internally by various Teradata projects for automation, testing, etc","archived":false,"fork":false,"pushed_at":"2017-09-26T16:51:42.000Z","size":320,"stargazers_count":40,"open_issues_count":11,"forks_count":22,"subscribers_count":19,"default_branch":"master","last_synced_at":"2025-03-22T19:12:05.830Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Teradata.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-09-30T13:01:53.000Z","updated_at":"2024-08-20T14:19:07.000Z","dependencies_parsed_at":null,"dependency_job_id":"dc148082-2023-4fa4-852a-10a18d61724d","html_url":"https://github.com/Teradata/docker-images","commit_stats":null,"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Teradata%2Fdocker-images","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Teradata%2Fdocker-images/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Teradata%2Fdocker-images/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Teradata%2Fdocker-images/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Teradata","download_url":"https://codeload.github.com/Teradata/docker-images/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247661769,"owners_count":20975116,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-06T04:18:08.628Z","updated_at":"2025-04-07T13:31:42.678Z","avatar_url":"https://github.com/Teradata.png","language":"Shell","readme":"# Teradata docker images\n\n## Docker Image Names\n\nThe docker images in this repository are expected to be given names of the form\nteradatalabs/cdh5-hive. The Dockerfile and other files needed to build the\nteradatalabs/cdh5-hive image are located in the directory\nteradatalabs/cdh5-hive.\n\nGenerally speaking, the images should *not* be built manually with docker\nbuild.\n\n## Building docker images\n\nThe docker images should be built using `make`. To build the docker image named\n`teradatalabs/cdh5-hive`, run `make teradatalabs/cdh5-hive`. Make will build\nthe image and its dependencies in the correct order.\n\nIf you are going to release an image, you should release it and all of its\ndependencies. Master and slave images should be built from the same chain of\nparent images. You can ensure that both are built from the same set of parent\nimages by running e.g. `make teradatalabs/cdh5-hive-master\nterdatalabs/cdh5-hive-slave`\n\nIf you want to build a base image and all the images depending on it,\nyou can use the `*.dependants` targets. E.g.\n\n```\nmake teradatalabs/cdh5-base.dependants\n```\n\nwill build the `cdh5-base` and all the images depending on it (transitively).\n\n## Releasing (pushing) docker image\n\nAll of the docker images in the repository share the same version number. This\nis because most of the images depend on a parent image that is also in the\nrepository (e.g. teradatalabs/hdp2.5-master is FROM teradatalabs/hdp2.5-base),\nor are meant to be used together in testing (teradatalabs/cdh5-hive-master and\nteradatalabs/cdh5-hive-slave).\n\nHaving all of the images on the same version number make troubleshooting easy:\nIff all of the docker images you are using have the same version number then\nthey are in a consistent state. \n\nThis means that we treat the repository as a single codebase that creates\nmultiple artifacts (Docker images) that all need to be released together. The\nMakefile uses [docker-release](https://github.com/kokosing/docker-release) to\nautomate this process and ensure that the images on dockerhub are in a\nconsistent state provided all of the push operations run to completion.\n\ndocker-release also handles tagging the images and repository appropriately so\nthat you can easily find the Dockerfile used to create an image starting from\njust the tags on a Docker image.\n\nBest practice for publishing a snapshot or release version is to use the Jenkins job.  Login to Jenkins and search for `docker-images`.  If you must publish a new version manually, follow these steps:\n\nTo release a snapshot version of the repository do the following\n\n1. `docker login`\n2. Verify in the Makefile that `VERSION` is set to something ending in -SNAPSHOT.\n3. `make snapshot`\n\nTo release a release (final) version of the repository do the following\n\n1. `docker login`\n2. Verify in the Makefile that `VERSION` is set to something *not* ending in -SNAPSHOT.\n3. `make release`\n\nTo release a snapshot or final version, you must log in to docker using the\ndocker `login` command.\n\n### Typical workflow\n\nNormally developers are working on a snapshot version of the next release, and\nthe `VERSION` macro in the Makefile should be set to a snapshot version such as\n35-SNAPSHOT. A typical workflow is as follows:\n\n1. Develop changes\n2. Commit changes\n3. `make snapshot` to push snapshot releases to dockerhub as needed\n4. Repeat as needed\n\nEventually, version 35-SNAPSHOT is ready for release. To release version 35, do\nthe following:\n\n1. Change `VERSION` to the release version: 35-SNAPSHOT -\u003e 35\n2. Commit the repository\n3. `make release` to push the images to dockerhub and tag the repository\n4. Change `VERSION` to the next snapshot version: 35 -\u003e 36-SNAPSHOT\n5. Commit the repository\n6. Continue developing as described above\n\n`make snapshot` does the following:\n\n* Updates the 'latest' tag for the image on dockerhub\n* Creates a tag for the image with the git hash of the git repository on dockerhub\n\n`make release` does the following:\n\n* Updates the 'latest' tag for the image on dockerhub\n* Creates a tag for the image on dockerhub with the git hash of the git repository\n* Creates a tag for the image on dockerhub with the $(VERSION) specified in the Makefile\n* Creates a tag in the git repository with the name release-$(VERSION)\n\n`docker-release` enforces several rules about the state of the repository when pushing to dockerhub:\n\n* For a snapshot or a release, the repository must be in a clean state (no uncommitted files)\n* For a release, the branch must be master\n\n## Upgrading Docker images for consumers\n\nFor a project that uses Travis for continuous integration, you can upgrade the\ndocker images used by the project using the following process.\n\n1. Develop locally, testing your changes\n2. When you are satisfied with your changes, run `make snapshot` to release a snapshot build to dockerhub.\n3. Create a branch of the dependent project\n4. Set the tag for the images on the project to the tag of the snapshot build\n5. Push the branch to github.\n6. Merge your changes into docker-images/master\n7. Update the release version as described above, and run `make release`\n8. Create a PR against upstream/master that uses the new release of the docker images.\n9. After Travis passes, merge the PR\n\n## Docker build arguments (ARG, --build-arg)\n\nDocker build arguments are documented in the [Dockerfile\nreference](https://docs.docker.com/engine/reference/builder/#/arg)\n\nArgs are used by specifying the ARG directive in a Dockerfile:\n\n```\nARG FOO\nRUN echo $FOO \u003e/etc/foo\n```\n\nThe value of FOO then needs to be set in the Makefile:\n\n```\nFOO := Docker images build on $(shell uname -s) are superior to all others.\n```\n\nNote that `docker build` does *not* allow the variable reference `$FOO` to be\nwritten `${FOO}` or `$(FOO)`. Further note that it won't warn you about this;\ninstead, you'll likely end up with an error later in the build or a broken\nimage.\n\n`docker build` won't let you pass `--build-arg`s that don't have a\ncorresponding key in the Dockerfile. This means that the build system can't\njust pass the union of all of the `--build-arg`s needed by every Dockerfile in\nthe repository. The build system handles this largely the same way it handles\nfiguring out what the correct dependency order is for building the images,\ndescribed below.\n\nBuild args with a default value are not handled at present. Feel free to add\nthat functionality in `flag.sh` if needed.\n\n## Java\n\nIndividual Dockerfiles shouldn't contain the URL for downloading Java, the name\nof the RPM, or the path that java gets installed in. Doing this makes upgrading\nJava across the repo a pain with a bunch of touch points.\n\nInstead, the build system exposes the [Docker build\narguments](https://docs.docker.com/engine/reference/builder/#/arg) `JDK_URL`,\n`JDK_RPM`, and `JDK_PATH`. These can be used in your Dockerfile as follows:\n\n```\nARG JDK_URL\nRUN wget $JDK_URL\n```\n\n## How the build system works.\n\nAt a high level, a docker image depends on two things:\n\n1. Its Dockerfile\n2. Its parent image, specified by the from FROM line in the Dockerfile.\n\nUsing the relative directory from the root of the repo as the image name, we\ncould, in principle, write a rule of the form\n\n```\nteradatalabs/foo: teradatalabs/foo/Dockerfile $(extract_parent teradatalabs/foo/Dockerfile)\n\tcd teradatalabs/foo \u0026\u0026 docker build -t teradatalabs/foo .\n```\n\nUsing automatic variables we could shorten that to the following:\n\n```\nteradatalabs/foo: $@/Dockerfile $(extract_parent $@/Dockerfile)\n\tcd $@ \u0026\u0026 docker build -t $@ .\n```\n\nThis is conceptually valid, but it doesn't work: Automatic variables aren't\navailable in the prerequisites. The solution to solve that is to use a\npattern rule:\n\n```\n$(images): %: %/Dockerfile $(extract_parent %/Dockerfile)\n\t...\n```\n\nThat almost works. Almost because you can't use the stem (%) in a [function\ncall](https://www.gnu.org/savannah-checkouts/gnu/make/manual/html_node/Pattern-Rules.html).\n\nInstead, we can use three features of make together to accomplish the same thing.\n\n1. You can specify the same target multiple times with different\ndependencies. Make will build all of the dependencies before running the\ncommands to build the target.\n2. you can use the include directive to tell make to include another file.\n3. If a file specified by an include directive doesn't exist, make will\nlook for a rule to create that file.\n\n```\nteradatalabs/foo: teradatalabs/foo_parent\nteradatalabs/foo: teradatalabs/foo/Dockerfile\n\t...\n```\n\nThe strategy is to include a separate file that specifies the dependency on\nthe parent image. This file isn't in the repo, so the Makefile has a rule to\nmake it from the image's Dockerfile. The second rule specifies the dependency\non the Dockerfile and builds the image using docker build.\n\n[Recursive Make Considered Harmful](http://lcgapp.cern.ch/project/architecture/recursive_make.pdf)\nexplains this technique in section 5.4 and applies it to C source files and the\n.h files they include. I've adapted it here.\n\nThe depend.sh script generates a .d file in $(DEPDIR) from the Dockerfile for\nthe image:\n\n```\n$(DEPDIR)/teradatalabs/foo.d: teradatalabs/foo/Dockerfile\n\t...\n```\n\nThe corresponding .d file will take one of two forms:\n\n1. if foo's parent is built from this repository\n\n   ```\n   teradatalabs/foo: teradatalabs/foo_parent\n   ```\n\n2. if foo's parent should be pulled from dockerhub\n\n   ```\n   teradatalabs/foo:\n   ```\n\nIn the first case, make now knows that foo_parent is a dependency of foo, and\nbuilds it first.\n\nIn the second case, we don't add a dependency for make, and docker itself is\nresponsible for pulling foo's parent from dockerhub as part of the docker build\nprocess.\n\nA major difference between the approach explained in Recursive Make\nConsidered Harmful is that depend.sh needs to know what images the repo knows\nhow to build so it can output the second form for parent images we *don't*\nknow how to build. We do this by passing in the names of all of the images we\nknow how to build.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fteradata%2Fdocker-images","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fteradata%2Fdocker-images","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fteradata%2Fdocker-images/lists"}