{"id":21471787,"url":"https://github.com/brianredbeard/gpget","last_synced_at":"2025-07-15T08:31:30.497Z","repository":{"id":57501137,"uuid":"49741045","full_name":"brianredbeard/gpget","owner":"brianredbeard","description":"A simple utility to safely retrieve arbitrary files.","archived":false,"fork":false,"pushed_at":"2016-10-18T17:43:46.000Z","size":30,"stargazers_count":65,"open_issues_count":6,"forks_count":5,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-09T00:41:31.447Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/brianredbeard.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-01-15T19:44:10.000Z","updated_at":"2025-02-28T03:02:51.000Z","dependencies_parsed_at":"2022-09-14T19:40:36.098Z","dependency_job_id":null,"html_url":"https://github.com/brianredbeard/gpget","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/brianredbeard/gpget","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brianredbeard%2Fgpget","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brianredbeard%2Fgpget/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brianredbeard%2Fgpget/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brianredbeard%2Fgpget/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/brianredbeard","download_url":"https://codeload.github.com/brianredbeard/gpget/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/brianredbeard%2Fgpget/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265419694,"owners_count":23761858,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-23T09:45:40.571Z","updated_at":"2025-07-15T08:31:30.202Z","avatar_url":"https://github.com/brianredbeard.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"## GPget \n\n### About\n\nThe purpose of this utility is to securely retrieve URLs for use in computing\nenvironments.  A design pattern has become to use the utility `curl` to\nretreive a remote endpoint and directly pipe this into a shell interpreter.\nWhile this provides for a distinct ease of use and the ability to change\nruntime image based server deployments, it is of dubious security.\n\n### Why?\n\nIncreasingly users are implementing services on untrustworthy networks. Let's\ntake the case of public utility computing as prophesized by John McCarthy (aka\nAmazon AWS, GCE, etc).  One of the core services provided by these services\nis that of \"object storage\", the ability to host a file an provide access to\nit via a services interface (generally `HTTP`).  While wonderful for building\nhighly scalable network services, it is problematic in that there is no\nattestation around access or modification to the files.  Surely one could hash\nevery file uploaded and verify the hashes, but when Alice is trying to talk to\nBob they can't rule out that Mallory is a state actor.  In this case, they need\na strong mechanism of attesting that their files have not been modified on\nstorage or in transit.  TLS can only protect against the _transport_ security\nnot the on disk case.  This it is desireable to take this to the lowest common\ndenominator.  Why not build a system even usable on _hostile_ networks?  This\nis the goal of *GPget*.\n\n#### Mechanism\n\nThrough a series of well formed URIs we attempt a series of HTTP GET requests\nto retrieve both a desired endpoint as well as a series of extensions used for\nthe attestation of that file.  The two standard extensions for these files are\n`.sig` and `.asc`.  `.sig` files are binary based GPG/PGP signatures while\n`.asc` files are Base64 encoded armored signatures.  *GPGget expects to\nretrieve the armored `.asc` extention.*  The signature files are \"detached\" \nand created in compliance with RFC 4880.\n\nThis utility will download those files and operate largely as one would expect\n`curl` to, only with the addition of GPG validation.  First, if the files fail\nto pass GPG validation, the utility will exit on a non-zero error code.  This\nis to ensure that a user can correctly operate using normal POSIX error\nhandling.  Next, as the files have passed validation, we achieve the next choice\nin the program.  If the user has supplied no additional arguments, the utility\nwill output the data stream to STDOUT.   If the user has supplied to argument\n`-O`, then the file will be written down to disk, and exit with a zero error \ncode.  If the file cannot be written to disk, the utility will exit non-zero. \n\nIn short, if there is any behavior which deviates from the basic pattern of \n\"request a URI, validate it cryptographically, and pass the data to the users\ndesired location\" the program will exit non-zero.\n\n#### Trust\n\nThe GPG trust model is patterened after the APPC container specification.\nOriginally GPG utilized the standard keychain mechanism generally used by\nmost PGP/GPG based utilities.  The reality was that the actual use pattern\nfollows much more closely to that of the CA trust model of openssl.  The\nresult is the ability to trust a series of signatures solely through the\nuse of file operations and without execution of any GPG/GPG2 binary directly.\n\n### Examples\n\n#### Authorized Keys\nGiven a situation where users want to provide access to a host without the full\nfeature set of a centralized AAA (authentication, authorization, and \naccounting) system a streamlined mechanism for securely providing this access\ncan be achieved with GPget.\n\nFirst, one would generate an ssh keypair with which to access a host:\n\n```\n$ ssh-keygen   -b 8192 -t ed25519 -C \"Deployment key for host foo.example.com\" -f  foo \nGenerating public/private ed25519 key pair.\nEnter passphrase (empty for no passphrase): \nEnter same passphrase again: \nYour identification has been saved in foo.\nYour public key has been saved in foo.pub.\nThe key fingerprint is:\nSHA256:agsS0ZY7Uh579d2NjXO1NSeUlzEU/8qFF5jQigzrhs8 Deployment key for host foo.example.com\nThe key's randomart image is:\n+--[ED25519 256]--+\n|           .. o*+|\n|   . . .    .o+.+|\n|  . *   = . .o.+=|\n|   = + o + o . BO|\n|  o = + S . . *.B|\n|   o + +     . * |\n|  . . *       o  |\n|   . o E         |\n|      .          |\n+----[SHA256]-----+\n```\n\nNext, the public key will be signed by the build system (or the security team\nto truely do it \"right\") with a detached signature.  Both armored ASCII and\nbinary signatures are supported, though armored is the default:\n\n```\n$ gpg -b -u jenkins@example.com -a  foo.pub \n$ cat foo.pub\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIM07Yybpo5KPALuqpTffHTyAvUiJclIiBFU6jAY4xGAO Deployment key for host foo.example.com\n$ cat foo.pub.asc \n-----BEGIN PGP SIGNATURE-----\nVersion: GnuPG v1\n\niQIcBAABAgAGBQJWnodYAAoJELlooAvCoMRn29QP+wbvQddEoXSfONow4zk/HgBM\n8PcxTRhAVsGgo77evjKE1TUWQvr4gxS5kX73mrnMlhnIseCCj2IOs7Pf8bqoXAlB\nq3v6aRGXCsvc2tN0TQtxi0qddcuc38ZCXkdUOJ+1bvOsdkyQ1pPfx3Ra9K/kadAO\n5EMvMjnZjbzDAkfr4SeGXZXbMZiNLSACoe5wB45hg+5XpwtEc0dJwyyl2JZmfrND\nP0AzxRLdZbXKmn0xjahBwp0UhojrXDYVsiZJDIBo6tW0NVDsuNFsKm81teLSh4Lj\nuItlVg45eVE3TbJeAQsm9/aIOCRBvMkP/XFV5wxcd1Bge4rnyyneHkozLyoAm9Pe\nl+bAmiJerCUjoeY4sNmj01gLN7QHAdEftaM0p8iyad8Eum8LZo2extYlfw1+nAbN\nRxToPmzdrO9Azfe/Q4OgwEOP3SHbKTRzI9vA2SaOpMF1IIUt0ziLc4l7hyJJ7Tk7\nk96/BmObDcGmj/bbuj9UsCcgIsOToO7NUjs3bnkPFUCJi7YXT+ZGu3L4NHebzAhl\nsXozQYuarUzn3JSF7Xn7NLiLXWH5hcoeyBLkGoZK+vv9w01L9e/9EN5kA/OROEf9\nh306yzg1fTRa59Ia9iysjmqr6hRF0glzbqNiG3QBf9ibspyxCs2MeVabsw8MM0p3\nhk9TcKBRhDDCrF23eLFV\n=8ouN\n-----END PGP SIGNATURE-----\n$ gpg --verify foo.pub.asc \ngpg: assuming signed data in `foo.pub'\ngpg: Signature made Tue 19 Jan 2016 10:58:32 AM PST using RSA key ID C2A0C467\ngpg: Good signature from \"Jenkins Build System (Example, Inc) \u003cjenkins@example.com\u003e\"\n```\n\nAt this point, the private key can be placed in escrow and the public key and \nsignature published to a generic web hosting service.  This could be anything\nfrom Amazon S3, Google GCS, GitHub, etc.\n\nFrom there the remote host could be deployed with a job that includes no public\nkeys for it's build user but instead a cron job which will run periodically,\nretrieve those keys from the untrusted storage, and yet cycle keys as needed.\n\nImagine the following processes happening on provisioning:\n\n```\n$ gpg --import B968A00BC2A0C467\ngpg: key 0xB968A00BC2A0C467: public key \"Jenkins Build System (Example, Inc) \u003cjenkins@example.com\u003e\" imported\ngpg: Total number processed: 1\ngpg:               imported: 1  (RSA: 1)\n```\n\nand a cron job calling the following command every five minutes\n\n```\n$ /usr/local/bin/gpget -k B968A00BC2A0C467 -o /home/nobody/.ssh/authorized_keys --url http://s3.aws.amazon.com/examplebucket/foo.pub\n```\n\nThis will continuously pull down that public key file, verify the GPG based\nprovinence of the file and only write it to the filesystem in the event that\nit can successfully both download AND verify the signature.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrianredbeard%2Fgpget","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbrianredbeard%2Fgpget","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbrianredbeard%2Fgpget/lists"}