{"id":20572173,"url":"https://github.com/mcaimi/ocp4-signature-verification","last_synced_at":"2025-08-30T12:16:20.339Z","repository":{"id":69446930,"uuid":"259626308","full_name":"mcaimi/ocp4-signature-verification","owner":"mcaimi","description":"A simple way to verify container images signature with CRI-O, skopeo and GPG","archived":false,"fork":false,"pushed_at":"2021-05-21T12:53:31.000Z","size":224,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-07-05T12:45:48.915Z","etag":null,"topics":["gpg","luajit","openresty","openshift-v4","redhat","signature-verification"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mcaimi.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-04-28T12:08:47.000Z","updated_at":"2022-02-02T17:47:07.000Z","dependencies_parsed_at":"2024-01-27T11:45:10.452Z","dependency_job_id":null,"html_url":"https://github.com/mcaimi/ocp4-signature-verification","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/mcaimi/ocp4-signature-verification","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcaimi%2Focp4-signature-verification","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcaimi%2Focp4-signature-verification/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcaimi%2Focp4-signature-verification/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcaimi%2Focp4-signature-verification/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mcaimi","download_url":"https://codeload.github.com/mcaimi/ocp4-signature-verification/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mcaimi%2Focp4-signature-verification/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":272847364,"owners_count":25003180,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-30T02:00:09.474Z","response_time":77,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gpg","luajit","openresty","openshift-v4","redhat","signature-verification"],"created_at":"2024-11-16T05:18:39.217Z","updated_at":"2025-08-30T12:16:20.275Z","avatar_url":"https://github.com/mcaimi.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"OPENSHIFT 4 IMAGE SIGNATURE VERIFICATION WIP DEMO\n==============================================================\n\nWith OpenShift 4, container image signatures can be verified at deploy time by configuring CRI-O to use GPG keys.\nThis repo contains a walkthrough/demo of this feature and a simple solution to signature storage. This demo will use:\n\n- Openshift 4 as the container platform\n- An external Sonatype Nexus instance to simulate a private docker registry\n- GPG to sign/verify container images\n- NGINX + LuaJIT as the application framework (OpenResty)\n\nSignatures can also be stored inside Nexus, so if this is the intended workflow, the NGINX signature server is not required.\nAlso, gpg signature verification is enabled for official RedHat repositories.\n\nPREREQUISITES\n-------------\n\n1) Install Openshift 4.x\n\n2) Generate a GPG Key that will be used to sign images\n\n.. code:: bash\n\n    # gpg --quick-gen-key demo@redhat.com\n    # gpg -k\n    [...]\n    pub   rsa2048 2020-04-24 [SC] [expires: 2022-04-24]\n      01164344435F9572F7B8B06D48790DBE02151245\n    uid           [ultimate] demo@redhat.com\n    sub   rsa2048 2020-04-24 [E]\n    [...]\n\n3) Export the public key to file\n\n.. code:: bash\n\n    # gpg --armor --export demo@redhat.com \u003e nexus-key.gpg\n\n4) Generate another GPG Key that will be used to simulate a wrongly signed image\n\n.. code:: bash\n\n    # gpg --quick-gen-key wrong@redhat.com\n\n5) Deploy an instance of Nexus\n\n.. code:: bash\n\n    # oc create -f components/nexus-deployment.yaml\n\n6) On the nexus web interface, create a new Hosted Docker repository.\n\n.. image:: img/nexus-docker-repo.png\n\n7) If you are using a self-signed certificate on the ingress controllers, the local nexus needs to be added to the list of `insecure`_ registries:\n\n.. code:: bash\n\n  # oc edit image.config.openshift.io/cluster\n\n.. code:: yaml\n\n  apiVersion: config.openshift.io/v1\n  kind: Image\n  metadata:\n    annotations:\n      release.openshift.io/create-only: \"true\"\n    creationTimestamp: 2020-05-05T05:21:57Z\n    generation: 2\n    name: cluster\n    resourceVersion: \"80728\"\n    selfLink: /apis/config.openshift.io/v1/images/cluster\n    uid: 5576bd84-9e0c-4d67-9498-9c8cf523cbd2\n  spec:\n    registrySources:\n      insecureRegistries:\n      - nexus-registry.apps.\u003cingress domain associated with the openshift cluster\u003e\n  status:\n    internalRegistryHostname: image-registry.openshift-image-registry.svc:5000\n\nThe MachineConfigOperator monitors that resource for differences and applies the new config when appropriate.\n\n8) If the image registry created on nexus needs authentication, a pull secret needs to be created and linked to the correct ServiceAccount\n\n.. code:: bash \n\n  # oc new-project signature-demo\n  # oc create secret docker-registry nexus-pull-secret --docker-server=nexus-registry.apps.\u003cingress domain associated with the openshift cluster\u003e --docker-username=\u003cusername\u003e --docker-password=\u003cpassword\u003e --docker-email=unused\n\nFor example, if the 'demo-sa' is used to deploy pods with a deploymentConfig, this pull secret needs to be linked to that SA:\n\n.. code:: bash\n\n  # oc create sa demo-sa\n  # oc secrets link demo-sa nexus-pull-secret --for=pull\n\nTo assign the pull secret to the 'default' service account (the SA that is used when no other is specified):\n\n.. code:: bash\n\n  # oc secrets link default nexus-pull-secret --for=pull\n\nCONFIGURE OPENSHIFT NODES\n-------------------------\n\nThis demo uses a local instance of Nexus as an external image repository. We want images coming from that repo to be signed and verified.\nWorker (and masters optionally) nodes in an OCP cluster need to be made aware of a new repo that requires signature verification.\n\nThe policy.json file will contain all repositories that need signature verification.\nFor example, the resulting policy.json file will look like this with the custom 'nexus-registry.apps.\u003cingress domain associated with the openshift cluster\u003e' repository added in:\n\n.. code:: json\n\n    {\n      \"default\": [\n        {\n          \"type\": \"insecureAcceptAnything\"\n        }\n      ],\n      \"transports\": {\n        \"docker\": {\n          \"registry.access.redhat.com\": [\n            {\n              \"type\": \"signedBy\",\n              \"keyType\": \"GPGKeys\",\n              \"keyPath\": \"/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release\"\n            }\n          ],\n          \"registry.redhat.io\": [\n            {\n              \"type\": \"signedBy\",\n              \"keyType\": \"GPGKeys\",\n              \"keyPath\": \"/etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release\"\n            }\n          ],\n          \"nexus-registry.apps.\u003cingress domain associated with the openshift cluster\u003e\": [\n            {\n              \"type\": \"signedBy\",\n              \"keyType\": \"GPGKeys\",\n              \"keyPath\": \"/etc/pki/rpm-gpg/nexus-key.gpg\"\n            }\n          ]\n        },\n        \"docker-daemon\": {\n          \"\": [\n            {\n              \"type\": \"insecureAcceptAnything\"\n            }\n          ]\n        }\n      }\n    }\n\nConfiguration files are automatically rendered with the provided 'gen-machineconfig.sh' script.\n\n1) Create a configuration file for every repo and fill in the address of the HTTP server that will host the signatures:\n\n.. code:: yaml\n\n    docker:\n        nexus-registry.apps.\u003cingress domain associated with the openshift cluster\u003e:\n            sigstore: https://signature.apps.\u003cingress domain associated with the openshift cluster\u003e/sigstore\n\nCreate a file like this for all custom/official repositories enumerated in the policy.json file and that need GPG signature verification.\n\n2) Generate the MachineConfig manifests with the provided script (under machineconfig/)\n\n.. code:: bash\n\n  # ./gen-machineconfig.sh -k /path/to/nexus-key.gpg -r /path/to/nexus-registry.apps.\u003cingress domain associated with the openshift cluster\u003e.yaml\n\nThis will create two MachineConfig manifest files under the ./rendered/ folder:\n\n.. code:: bash\n\n  # oc create -f 02-master-rh-registry-trust.yaml\n  # oc create -f 02-worker-rh-registry-trust.yaml\n\nAfter a while both configuration will be applied to the cluster.\n\n.. code:: bash\n\n  # oc get machineconfigpool\n  NAME      CONFIG                                             UPDATED   UPDATING   DEGRADED   MACHINECOUNT   READYMACHINECOUNT   UPDATEDMACHINECOUNT   DEGRADEDMACHINECOUNT   AGE\n  master    rendered-master-36f5d702f485cde72df754013e17937f   True      False      False      3              3                   3                     0                      4d5h\n  worker    rendered-worker-ec7bab1743d5d2a88bed9cf1280ff9f1   True      False      False      3              3                   3                     0                      4d5h\n\nDEPLOY THE SIGNATURE SERVER\n---------------------------\n\nContainer images signatures are served by a simple HTTP server (nginx) with a couple service APIs baked in.\n\n1) Create a new project on OCP and set up the correct SCC for the sigserver service account\n\n.. code:: bash\n\n  # oc new-project signature-server\n  # oc adm policy add-scc-to-user anyuid system:serviceaccount:signature-server:signature-sa\n\n2) Create the virtual host config map:\n\n.. code:: bash\n\n  # oc create configmap nginx-sigstore-vhost --from-file=nginx/sigstore.conf\n\n3) Create the API configmap\n\n.. code:: bash\n\n  # oc create configmap lua-api-sources --from-file=api/context_body.lua --from-file=api/signature_upload.lua --from-file=api/filesystem.lua\n\n4) Deploy the signature server\n\n.. code:: bash\n\n  # oc create -f components/signature-server-deployment.yaml\n\nFETCH AND TEST IMAGE SIGNATURE\n------------------------------\n\nThis test makes use of three different small container images, to demonstrate these three use cases:\n\n- A Correctly Signed image (signed with the approved and configured GPG key)\n- An Image that has no signature\n- An Images that has been signed with an unknown/wrong GPG key\n\nSkopeo needs to be configured to store signatures in a known path, so that these can later be uploaded to a signature store:\n\n.. code:: bash\n\n  [...]\n  # This is the default signature write location for docker registries.\n  default-docker:\n  #  sigstore: file:///var/lib/containers/sigstore\n    sigstore-staging: file:///tmp/sigstore\n  [...]\n\nthe 'sigstore-staging' parameter is used by skopeo. After a successful sign operation, the signature is stored under that path:\n\n.. code:: bash\n\n  # tree /tmp/sigstore\n  /tmp/sigstore/\n  └── docker\n      └── busybox@sha256=a2490cec4484ee6c1068ba3a05f89934010c85242f736280b35343483b2264b6\n          └── signature-1\n\n1) Upload an image without signature to nexus\n\n.. code:: bash\n\n  # skopeo copy --dest-creds=\u003cusername\u003e:\u003cpassword\u003e docker://docker.io/library/alpine:latest docker://nexus-registry.apps.\u003cingress domain associated with the openshift cluster\u003e/docker/alpine:unsigned\n\n2) Upload an image signed with the wrong key to nexus\n\n.. code:: bash\n\n  # skopeo copy --dest-creds=\u003cusername\u003e:\u003cpassword\u003e --sign-by wrong@email.com docker://docker.io/library/busybox:latest docker://nexus-registry.apps.\u003cingress domain associated with the openshift cluster\u003e/docker/busybox:wrongsig\n\n3) Upload an image signed with the correct gpg key to nexus\n\n.. code:: bash\n\n  # skopeo copy --dest-creds=\u003cusername\u003e:\u003cpassword\u003e --sign-by demo@redhat.com docker://docker.io/library/centos:latest docker://nexus-registry.apps.\u003cingress domain associated with the openshift cluster\u003e/docker/centos:signed\n\nAfter that, in this third case, the image signature needs to be uploaded to the signature server.\n\nUPLOAD SIGNATURE TO KEYSTORE\n----------------------------\n\nUploading signature is achieved by calling the /upload API endpoint served by the signature server. All parameters need to be base64-encoded.\nThere is only one POST method implemented and that accepts a JSON payload:\n\n.. code:: json\n\n  {\n    \"repoName\": \"base64-encoded name of the repo on the remote docker registry\",\n    \"layerId\": \"base64-encoded sha digest of the signed container layer\",\n    \"signatureData\": \"base64-encoded signature of the image layer\"\n  }\n\nAn helper script is provided under jenkins-agents/signer-agent/scripts:\n\n.. code:: bash\n\n  # ./clients/signature-upload.py -r https://signature.apps.\u003cingress domain associated with the openshift cluster\u003e/upload -a /tmp/sigstore/docker/busybox@sha256=a2490cec4484ee6c1068ba3a05f89934010c85242f736280b35343483b2264b6/signature-1\n\nthis script takes the absolute path to the local signature of the container, builds the json payload and sends that to the signature server via a POST HTTP call.\nAlso, this script makes use of the python3 interpreter so a linux distro that supports at least:\n\n#) A fairly recent version of python3\n#) The python-requests library for python3\n\nis absolutely mandatory.\n\nUPLOAD SIGNATURES TO NEXUS (OPTIONAL)\n-------------------------------------\n\nRAW Repositories in Nexus3 can also host image signature files, so instead of deploying a separate signature server, the same Nexus used to store container images can be used to store signatures too.\n\n1) Create a RAW hosted repository called 'sigstore'\n\n.. image:: img/raw_repository.png\n\n2) Enable anonymous access\n\n.. image:: img/anonymous_access.png\n\nThis is needed since in this demo CRI-O is configured without authentication support. Keep in mind that upload on the other hand *needs authentication*.\n\n3) Sign and upload the image as shown in previous paragraphs and then upload the signature to nexus\n\n.. code:: bash\n\n  # ./clients/signature-upload.py -r https://nexus.apps.\u003cingress domain associated with the openshift cluster\u003e -a /tmp/sigstore/docker/busybox@sha256=a2490cec4484ee6c1068ba3a05f89934010c85242f736280b35343483b2264b6/signature-1 --no-verify --nexus -s sigstore -u \u003cusername\u003e -p \u003cpassword\u003e\n\n4) Update the repository configuration to use Nexus instead of the HTTP signature server and update the MachineConfig manifests:\n\n.. code:: yaml\n\n    docker:\n        nexus-registry.apps.\u003cingress domain associated with the openshift cluster\u003e:\n            sigstore: https://nexus.apps.\u003cingress domain associated with the openshift cluster\u003e/repository/sigstore\n\nTESTING SIGNATURE VERIFICATION\n------------------------------\n\n1) Create the demo deploymentconfig\n\n.. code:: bash\n\n  # oc project signature-demo\n  # oc create -f components/demo-deployment.yaml\n\n2) Check out the \"unsigned\" container:\n\n.. code:: bash\n\n  # oc describe pod demo-unsigned-c5d8dddf6-5lkbs\n  [...]\n  Events:\n  Type     Reason     Age              From                                                 Message\n  ----     ------     ----             ----                                                 -------\n  Normal   Scheduled  \u003cunknown\u003e        default-scheduler                                    Successfully assigned signature-server/demo-unsigned-c5d8dddf6-5lkbs to ip-10-0-166-156.us-east-2.compute.internal\n  Normal   Pulling    10s              kubelet, ip-10-0-166-156.us-east-2.compute.internal  Pulling image \"nexus-registry.apps.ocp4.sandbox595.opentlc.com/docker/alpine:unsigned\"\n  Warning  Failed     10s              kubelet, ip-10-0-166-156.us-east-2.compute.internal  Failed to pull image \"nexus-registry.apps.ocp4.sandbox595.opentlc.com/docker/alpine:unsigned\": rpc error: code = Unknown desc = Source image rejected: A signature was required, but no signature exists\n  Warning  Failed     10s              kubelet, ip-10-0-166-156.us-east-2.compute.internal  Error: ErrImagePull\n  Normal   BackOff    8s (x2 over 9s)  kubelet, ip-10-0-166-156.us-east-2.compute.internal  Back-off pulling image \"nexus-registry.apps.ocp4.sandbox595.opentlc.com/docker/alpine:unsigned\"\n  Warning  Failed     8s (x2 over 9s)  kubelet, ip-10-0-166-156.us-east-2.compute.internal  Error: ImagePullBackOff\n\nThe deployment is refused because images from nexus-registry need to be signed, but no signature has been uploaded to the sigstore for this image\n\n3) Check out the \"wrongly signed\" container:\n\n.. code:: bash\n\n  # oc describe pod demo-wrong-signature-68fb74b784-7tqmb\n  [...]\n    Events:\n  Type     Reason     Age                From                                                 Message\n  ----     ------     ----               ----                                                 -------\n  Normal   Scheduled  \u003cunknown\u003e          default-scheduler                                    Successfully assigned signature-server/demo-wrong-signature-68fb74b784-7tqmb to ip-10-0-166-156.us-east-2.compute.internal\n  Normal   BackOff    17s (x2 over 42s)  kubelet, ip-10-0-166-156.us-east-2.compute.internal  Back-off pulling image \"nexus-registry.apps.ocp4.sandbox595.opentlc.com/docker/busybox:wrongsig\"\n  Warning  Failed     17s (x2 over 42s)  kubelet, ip-10-0-166-156.us-east-2.compute.internal  Error: ImagePullBackOff\n  Normal   Pulling    5s (x3 over 43s)   kubelet, ip-10-0-166-156.us-east-2.compute.internal  Pulling image \"nexus-registry.apps.ocp4.sandbox595.opentlc.com/docker/busybox:wrongsig\"\n  Warning  Failed     5s (x3 over 43s)   kubelet, ip-10-0-166-156.us-east-2.compute.internal  Failed to pull image \"nexus-registry.apps.ocp4.sandbox595.opentlc.com/docker/busybox:wrongsig\": rpc error: code = Unknown desc = Source image rejected: Invalid GPG signature: gpgme.Signature{Summary:128, Fingerprint:\"4F06789A5C76861E\", Status:gpgme.Error{err:0x9}, Timestamp:time.Time{wall:0x0, ext:63723658926, loc:(*time.Location)(0x55f9f39502a0)}, ExpTimestamp:time.Time{wall:0x0, ext:62135596800, loc:(*time.Location)(0x55f9f39502a0)}, WrongKeyUsage:false, PKATrust:0x0, ChainModel:false, Validity:0, ValidityReason:error(nil), PubkeyAlgo:1, HashAlgo:8}\n  Warning  Failed     5s (x3 over 43s)   kubelet, ip-10-0-166-156.us-east-2.compute.internal  Error: ErrImagePull\n\nThe deployment is refused because although the image is correctly signed, the signature cannot be verified because the signer private key used to sign the image does not match the public key used to verify the signature.\n\n4) Check out the \"correctly signed\" container:\n\n.. code:: bash\n\n  # oc describe pod demo-signed-6c784b5957-4gpt7\n  [...]\n  Events:\n  Type    Reason     Age        From                                                 Message\n  ----    ------     ----       ----                                                 -------\n  Normal  Scheduled  \u003cunknown\u003e  default-scheduler                                    Successfully assigned signature-server/demo-signed-6c784b5957-4gpt7 to ip-10-0-166-156.us-east-2.compute.internal\n  Normal  Pulling    14s        kubelet, ip-10-0-166-156.us-east-2.compute.internal  Pulling image \"nexus-registry.apps.ocp4.sandbox595.opentlc.com/docker/centos:signed\"\n  Normal  Pulled     14s        kubelet, ip-10-0-166-156.us-east-2.compute.internal  Successfully pulled image \"nexus-registry.apps.ocp4.sandbox595.opentlc.com/docker/centos:signed\"\n  Normal  Created    13s        kubelet, ip-10-0-166-156.us-east-2.compute.internal  Created container pause\n  Normal  Started    13s        kubelet, ip-10-0-166-156.us-east-2.compute.internal  Started container pause\n\nThis deployment is approved because the signature is correctly found on the sigstore and the verification succeeded with the configured public key.\n\nTODO\n----\n\n#) Integrate into a Jenkins pipeline\n#) Make the scripts/manifests more generically usable, as for example domains are for now hardcoded in code.\n#) Remove shell scripts, migrate to Helm\n#) Improve documentation\n\n.. _insecure: https://docs.openshift.com/container-platform/4.3/openshift_images/image-configuration.html\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmcaimi%2Focp4-signature-verification","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmcaimi%2Focp4-signature-verification","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmcaimi%2Focp4-signature-verification/lists"}