{"id":19643208,"url":"https://github.com/goethite/gostint","last_synced_at":"2025-04-28T13:30:40.839Z","repository":{"id":33859887,"uuid":"140021657","full_name":"goethite/gostint","owner":"goethite","description":"A Shallow RESTful api for container based automation with Ansible, Terraform, ... with Hashicorp Vault AppRole Authentication and End-to-End Encryption - https://goethite.github.io/gostint/","archived":true,"fork":false,"pushed_at":"2022-12-30T04:24:04.000Z","size":8478,"stargazers_count":5,"open_issues_count":15,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-14T19:52:56.653Z","etag":null,"topics":["ansible","api","approle","automation","containers","devops","docker","encryption","golang","hashicorp-vault","helm","kubectl","kubernetes","powershell","rest","restful","secrets","tamperproofing","terraform","vault"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/goethite.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-07-06T19:13:20.000Z","updated_at":"2024-10-02T14:08:53.000Z","dependencies_parsed_at":"2023-01-15T03:15:46.578Z","dependency_job_id":null,"html_url":"https://github.com/goethite/gostint","commit_stats":null,"previous_names":[],"tags_count":42,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/goethite%2Fgostint","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/goethite%2Fgostint/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/goethite%2Fgostint/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/goethite%2Fgostint/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/goethite","download_url":"https://codeload.github.com/goethite/gostint/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251319747,"owners_count":21570450,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ansible","api","approle","automation","containers","devops","docker","encryption","golang","hashicorp-vault","helm","kubectl","kubernetes","powershell","rest","restful","secrets","tamperproofing","terraform","vault"],"created_at":"2024-11-11T14:19:34.922Z","updated_at":"2025-04-28T13:30:40.788Z","avatar_url":"https://github.com/goethite.png","language":"Go","readme":"# gostint - A Shallow RESTful api for Ansible, Terraform ...\n... and basically anything you would like to run as jobs in docker containers.\nAuthenticated and (optionally) end-to-end encrypted with Hashicorp Vault with\nSecret Injection\n* https://goethite.github.io/gostint/\n\n\u003e gostint:\n\u003e : _stint - an allotted amount or piece of work_\n\n| ![](https://img.shields.io/github/release/goethite/gostint.svg) | ![](https://img.shields.io/github/license/goethite/gostint.svg) |\n|-|-|\n\nGoal is to be a Highly Available and Scaleable Secure API for automation.\n\nSee [Concept Ideas](docs/Concept_Ideas.md)\n\nAt this stage this project is a MVP and under development / review...\n\nPrebuilt releases are available [here](https://github.com/goethite/gostint/releases).\n\nSee [build_test_dev script](./build_test_against_dev.sh) for example starting the gostint docker container with the instances of Vault and MongoDb running in the vagrant container.\n\nSee [bats tests folder](tests/bats) for example `curl` command based BATs tests, that\ndemo driving the gostint api to run a selection of Docker container based jobs.\nJSON jobs used in these tests are in the respective [tests](tests/) files.\n\n* [Dev Notes](docs/devnotes.md)\n* [Job States](docs/jobstates.md)\n* [Brainstorming job sequence diagrams](docs/jobsequence.md)\n\n## Features\n* Integrated with Hashicorp Vault's AppRole Authentication.\n* Optionally consume Hashicorp Vault's Transit end-to-end\n  encryption, Cubbyhole and Token Wrapping if routing requests through\n  untrusted networks (e.g. where TLS end-to-end encryption is not available).\n* If TLS encryption is unbroken a much simpler way to consume the API is\n  available.\n* Secrets in Vault can be referenced in a job request, which are then injected\n  into the job's running container.\n* Additional content can be flexibly injected into the job container from the\n  json request.\n* Can run any job in any required docker image, e.g. Ansible, Terraform, Busybox,\n  Powershell, and the versions of the job execution containers can be pinned.\n* Serialisation queues are dynamic and created on the fly.\n\n## Usage\n\n### Prerequisites\n1. A MongoDB service\n\n2. A Hashicorp Vault service\nSee test setup in [scripts/init_vault.sh](scripts/init_vault.sh) for example of enabling the MongoDB Secret Engine in Vault.\n\n3. SSL Key and Certificate for gostint - `key.pem` and `cert.pem` stored in persistent volume shown below as `/srv/gostint-1/etc`\n\n### Running the gostint docker container\nA very basic setup for a single instance of gostint:\n```bash\n# point to your vault's url\nVAULT_ADDR=\"${VAULT_ADDR:-https://your.vault.host:8200}\"\n\n# login to the vault - using your chosen authentication scheme in vault\nvault login # to get a \u003ctoken\u003e\n\n# Request a MongoDB secret engine token for gostint to request an ephemeral\n# time-bound username/password pair.\n# TODO: this token is deprecated - Use Approle instead (gostint-run)\n# token=$(curl -s \\\n#   --request POST \\\n#   --header 'X-Vault-Token: \u003ctoken\u003e' \\\n#   --data '{\"policies\": [\"gostint-mongodb-auth\"], \"ttl\": \"10m\", \"num_uses\": 2}' \\\n#   ${VAULT_ADDR}/v1/auth/token/create | jq .auth.client_token -r)\n\n# Get gostint's AppRole RoleId from the Vault\nroleid=`curl -s --header 'X-Vault-Token: root' \\\n  ${VAULT_ADDR}/v1/auth/approle/role/gostint-role/role-id | jq .data.role_id -r`\n\n# Run gostint\ndocker run --init -d \\\n  --name gostint -p 3232:3232 \\\n  --privileged=true \\\n  -v /srv/gostint-1/etc:/var/lib/gostint \\\n  -e VAULT_ADDR=\"$VAULT_ADDR\" \\\n  -e GOSTINT_ROLEID=\"$roleid\" \\\n  -e GOSTINT_RUN_ROLEID=\"$runroleid\" \\\n  -e GOSTINT_RUN_SECRETID=\"$runsecretid\" \\\n  -e GOSTINT_DBURL=your-db-host:27017\n  goethite/gostint\n```\n\n### Going HA and Scalable with gostint\nSee [gostint-helm](https://github.com/goethite/gostint-helm) for (a work-in-progress)\nPoC HA deployment of gostint using mongodb, consul and vault on kubernetes.\n\n### gostint-client\nA sister project called [gostint-client](https://github.com/goethite/gostint-client)\nis also available to simplify the client side integrations with Hashicorp Vault\nand drive the [gostint api](https://goethite.github.io/gostint/docs/1100_api_v1_job/).\n\n### Enabling the gostint UI\nTo enable the experimental web UI in gostint, simply pass it `GOSTINT_UI=1`:\n```bash\n... GOSTINT_UI=1 gostint\n```\nAccess the UI at https://127.0.0.1:3232\n\n## Developer Guide\n\nDevelopment and testing is done in a Vagrant/Docker environment:\n```bash\n$ vagrant up\n...\n$ vagrant ssh\n```\nThe environment should already be running an instance of MongoDB and Hashicorp Vault:\n```bash\nvagrant@2c6839c78fbd:~$ ps -ef\nUID         PID   PPID  C STIME TTY          TIME CMD\nroot          1      0  0 11:09 ?        00:00:00 /usr/sbin/sshd -D -e\nroot       2498      1  0 11:10 ?        00:00:02 dockerd -s vfs\nroot       2526   2498  1 11:10 ?        00:00:04 containerd --config /var/run/docker/containerd/containerd.toml --log-level info\nroot       3256      1  1 11:10 ?        00:00:05 mongod --config /etc/mongod.conf --fork --smallfiles --auth --bind_ip 0.0.0.0\nroot       3309      1  0 11:10 ?        00:00:02 vault server -dev -dev-root-token-id=root -dev-listen-address=0.0.0.0:8200\nroot       3608      1  0 11:15 ?        00:00:00 sshd: vagrant [priv]\nvagrant    3610   3608  0 11:15 ?        00:00:00 sshd: vagrant@pts/0\nvagrant    3611   3610  0 11:15 pts/0    00:00:00 -bash\nvagrant    3629   3611  0 11:17 pts/0    00:00:00 ps -ef\n```\nNotice it is also running an instance of Docker-in-Docker (the vagrant instance\nruns the docker container in `privileged` mode to support this).\n\nChange to the gostint source folder (mapped by vagrant from your gostint git\nclone folder):\n```bash\nvagrant@2c6839c78fbd:~$ cd go/src/github.com/gbevan/gostint\n```\n\nand run `godo` to build and start the gostint application:\n```bash\n~/go/src/github.com/gbevan/gostint$ godo\nSuccess! You are now authenticated. The token information displayed below\nis already stored in the token helper. You do NOT need to run \"vault login\"\nagain. Future Vault requests will automatically use this token.\n\nKey                  Value\n---                  -----\ntoken                root\ntoken_accessor       4mOso4ZzgZ9PR5wjDwhM2YiK\ntoken_duration       ∞\ntoken_renewable      false\ntoken_policies       [\"root\"]\nidentity_policies    []\npolicies             [\"root\"]\nSuccess! Data written to: secret/my-secret\nSuccess! Data written to: secret/my-form\nKey              Value\n---              -----\ncreated_time     2018-11-24T11:24:21.632236029Z\ndeletion_time    n/a\ndestroyed        false\nversion          3\nKey              Value\n---              -----\ncreated_time     2018-11-24T11:24:21.665699192Z\ndeletion_time    n/a\ndestroyed        false\nversion          3\ngettoken\u003emocksecrets 187ms\n7lbKwKMayAkJnwUmHmnmLgge\ndefault\u003egettoken 12ms\ndefault 317ms\n\n                              |   _)         |\n           _` |   _ \\    __|  __|  |  __ \\   __|\n          (   |  (   | \\__ \\  |    |  |   |  |\n         \\__, | \\___/  ____/ \\__| _| _|  _| \\__|\n         |___/\n\nCopyright 2018 Graham Lee Bevan \u003cgraham.bevan@ntlworld.com\u003e\n               Licensed under the GNU GPLv3\n\nhttps://goethite.github.io/gostint/\nhttps://github.com/goethite\n\n2018/11/24 11:24:22 INFO: Starting gostint...\n2018/11/24 11:24:22 INFO: gostint listening on https port 3232\n```\n\n`godo` can also run in `watch` mode, so it automatically restarts when you make\nchanges to the code:\n```bash\n~/go/src/github.com/gbevan/gostint$ godo --watch\n```\n\nTo run the BATS test suite (in another terminal session):\n```bash\n~/go/src/github.com/gbevan/gostint$ godo test\n\n***************************\n*** Starting BATS Tests ***\n***************************\n...\n```\n\n### Build the UI and run godo\n```bash\n~/go/src/github.com/gbevan/gostint/ui$ npm run godo\n```\n\n## LICENSE - GPLv3\n\n```\nCopyright 2018-2019 Graham Lee Bevan \u003cgraham.bevan@ntlworld.com\u003e\n\ngostint is free software: you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation, either version 3 of the License, or\n(at your option) any later version.\n\ngostint is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License\nalong with gostint.  If not, see \u003chttps://www.gnu.org/licenses/\u003e.\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoethite%2Fgostint","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgoethite%2Fgostint","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoethite%2Fgostint/lists"}