{"id":13582452,"url":"https://github.com/smallstep/step-sds","last_synced_at":"2025-04-07T07:12:00.565Z","repository":{"id":37891993,"uuid":"179133177","full_name":"smallstep/step-sds","owner":"smallstep","description":"🔭 Secret discovery service (SDS): simplifying certificate management for relying parties (such as Envoy)","archived":false,"fork":false,"pushed_at":"2025-03-24T17:08:56.000Z","size":12492,"stargazers_count":74,"open_issues_count":2,"forks_count":12,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-03-31T06:02:57.072Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/smallstep.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-04-02T18:10:10.000Z","updated_at":"2025-03-24T17:09:01.000Z","dependencies_parsed_at":"2023-02-16T10:01:45.805Z","dependency_job_id":"f01664be-ee80-4267-a4b5-45fc7e52c755","html_url":"https://github.com/smallstep/step-sds","commit_stats":null,"previous_names":[],"tags_count":28,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/smallstep%2Fstep-sds","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/smallstep%2Fstep-sds/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/smallstep%2Fstep-sds/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/smallstep%2Fstep-sds/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/smallstep","download_url":"https://codeload.github.com/smallstep/step-sds/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247608153,"owners_count":20965952,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T15:02:43.756Z","updated_at":"2025-04-07T07:12:00.544Z","avatar_url":"https://github.com/smallstep.png","language":"Go","funding_links":[],"categories":["Documentation","Go"],"sub_categories":[],"readme":"# step-sds\nThe [secret discovery service (SDS)](https://www.envoyproxy.io/docs/envoy/latest/configuration/secret) simplifies certificate management\nand was originally created by the Envoy project to provide a flexible API to deliver secrets/certificates to the Envoy proxy.\n\nStep SDS server implements the server-side API of SDS which pushes certificates to the client. Both mTLS and Unix Domain Sockets\nconfiguration are supported. Use the one that better suits your environment/requirements.\n\n## mTLS initialization\n\n### Using step-sds\n\nTo use mTLS between [Envoy](https://www.envoyproxy.io) and our SDS server we need to initialize a PKI running with `step-sds init`. We will need the destination url and root certificate of your CA ([step certificates](https://github.com/smallstep/certificates)).\n\n```sh\n$ step-sds init --ca-url https://ca.smallstep.com:9000 --root ~/.step/certs/root.crt\n✔ What would you like to name your new PKI? (e.g. SDS): SDS\n✔ What do you want your PKI password to be? [leave empty and we'll generate one]:\n✔ What address will your new SDS server listen at? (e.g. :443): :8443\n✔ What DNS names or IP addresses would you like to add to your SDS server? (e.g. sds.smallstep.com[,1.1.1.1,etc.]): sds.smallstep.com\n✔ What would you like to name your SDS client certificate? (e.g. envoy.smallstep.com): envoy.smallstep.com\n✔ What do you want your certificates password to be? [leave empty and we'll generate one]:\n✔ Key ID: jO37dtDbku-Qnabs5VR0Yw6YFFv9weA18dp3htvdEjs (mariano@smallstep.com)\n\n✔ Root certificate: /home/user/.step/sds/root_ca.crt\n✔ Root private key: /home/user/.step/sds/root_ca_key\n✔ Intermediate certificate: /home/user/.step/sds/intermediate_ca.crt\n✔ Intermediate private key: /home/user/.step/sds/intermediate_ca_key\n✔ SDS certificate: /home/user/.step/sds/sds_server.crt\n✔ SDS private key: /home/user/.step/sds/sds_server_key\n✔ SDS client certificate: /home/user/.step/sds/sds_client.crt\n✔ SDS client private key: /home/user/.step/sds/sds_client_key\n✔ SDS configuration: /home/user/.step/config/sds.json\n\nYour PKI is ready to go.\nYou can always generate new certificates or change passwords using step.\n```\n\nThe `init` command will generate a root and intermediate certificate, with both\nkeys encrypted using the same password. And a server certificate for the\nstep-sds (sds_server.crt) and a client certificate (sds_client.crt) for Envoy to\nbe used to connect to the SDS server via mTLS. The SDS server and client keys will be\nencrypted with own password separate from the intermediate/root keys. `init` will also\ngenerate an initial configuration file. All files generated will be stored in your `STEPPATH` (just run `step path` to know where).\n\nIf you want to change the passwords or create your own PKI you can leverage the corresponding subcommands available in [step CLI](https://github.com/smallstep/cli).\n\n### Using step CLI\n\nAs we mention before we can use [step CLI](https://github.com/smallstep/cli) in lieu of\nthe `init`-flow. Assuming that the SDS is running on sds.smallstep.com and we name the\nenvoy client certificate as envoy.smallstep.com we can just run:\n\n```sh\n# Root and intermediate\nstep certificate create --profile root-ca \"Smallstep SDS Root CA\" root.crt root.key\nstep certificate create --profile intermediate-ca --ca root.crt --ca-key root.key \"Smallstep SDS Intermediate CA\" int.crt int.key\n\n# Step SDS\nstep certificate create --profile leaf --ca int.crt --ca-key int.key --no-password --insecure --not-after 87600h sds.smallstep.com sds.pem sds.key\nstep certificate bundle sds.pem int.crt sds.crt\n\n# Envoy\nstep certificate create --profile leaf --ca int.crt --ca-key int.key --no-password --insecure --not-after 87600h envoy.smallstep.com envoy.pem envoy.key\nstep certificate bundle envoy.pem int.crt envoy.crt\n```\n\n## Running the SDS server\n\nWith the PKI and configuration file ready, we can run the SDS server:\n\n```sh\n$ bin/step-sds run ~/.step/config/sds.json\nPlease enter the password to decrypt the provisioner key:\nPlease enter the password to decrypt /Users/mariano/.step/sds/sds_server_key:\nINFO[0002] Serving at tcp://[::]:8443 ...                grpc.start_time=\"2019-04-11T19:19:37-07:00\"\n```\n\nBy default it will ask you for the password to decrypt the provisioner key, and for\nthe certificate key password (if encrypted). You can avoid prompts using the `--password-file`\nand `--provisioner-password-file` flags.\n\n```\n$ bin/step-sds run ~/.step/config/sds.json --password-file /run/secrets/key.password --provisioner-password-file /run/secrets/provisioner.password\nINFO[0000] Serving at tcp://[::]:8443 ...                grpc.start_time=\"2019-04-11T19:21:59-07:00\"\n```\n\nAlternatively, to avoid interactive prompts, you can always specify passwords in the `sds.json` config file:\n\n```json\n{\n   \"network\": \"tcp\",\n   \"address\": \":8443\",\n   \"root\": \"/home/user/.step/sds/root_ca.crt\",\n   \"crt\": \"/home/user/.step/sds/sds_server.crt\",\n   \"key\": \"/home/user/.step/sds/sds_server_key\",\n   \"password\": \"[my-certificate-key-password]\",\n   \"authorizedIdentity\": \"envoy.smallstep.com\",\n   \"authorizedFingerprint\": \"8597a5d0b86f4a630f64fbb903b613ceb04756319a156bb6a6faed95394040ff\",\n   \"provisioner\": {\n      \"issuer\": \"mariano@smallstep.com\",\n      \"kid\": \"jO37dtDbku-Qnabs5VR0Yw6YFFv9weA18dp3htvdEjs\",\n      \"ca-url\": \"https://ca.smallstep.com:9000\",\n      \"root\": \"/home/user/.step/certs/root_ca.crt\",\n      \"password\": \"[my-provisioner-password]\"\n   },\n   \"logger\": {\n      \"format\": \"text\"\n   }\n}\n```\n\nAnd then just:\n\n```sh\n$ bin/step-sds run ~/.step/config/sds.json\nINFO[0000] Serving at tcp://[::]:8443 ...                grpc.start_time=\"2019-04-11T19:24:09-07:00\"\n```\n\nSDS clients (such as Envoy) can connect to the server via UNIX domain socket.\nIf you decide to use UNIX domain sockets the sds.json configuration file will\nlook different as it won't be necessary to configure TLS certificates. Instead,\nyou will only need to set the right network type (`unix`), address (file path\nfor socket) and a provisioner configured in your certificates CA:\n\n```json\n{\n    \"network\": \"unix\",\n    \"address\": \"/tmp/sds.unix\",\n    \"provisioner\": {\n       \"issuer\": \"sds@smallstep.com\",\n       \"kid\": \"oA1x2nV3yClaf2kQdPOJ_LEzTGw5ow4r2A5SWl3MfMg\",\n       \"ca-url\": \"https://ca:9000\",\n       \"root\": \"/home/user/.step/certs/root_ca.crt\"\n    },\n    \"logger\": {\n       \"format\": \"text\"\n    }\n }\n ```\n\n## Docker-Compose example\n\nIn [examples/docker](examples/docker) directory you'll find a docker-compose\nexample that initializes a CA, a SDS server, and Envoy proxying request to two\ndifferent servers, `frontend` \u0026 `backend` respectively. The SDS `init`-flow will\ngenerate certificates and send them to Envoy, the CommonName and DNS names of\nthe certificates will be specified by the `tls_certificate_sds_secret_configs`\nname in the [envoy configuration](examples/docker/envoy/server.yaml). In our\nexample we are using `hello.smallstep.com` for the `frontend` server and\n`internal.smallstep.com` for the `backend` server. The use of a client certificate\nto access the backend server is mandatory. This certificate must be signed by\nthe CA server.\n\nAssuming a docker daemon is running you can bring up the example running following\ncommands inside the main `step-sds` directory:\n\n```sh\nmake docker\ncd examples/docker/\ndocker-compose up\n```\n\nOnce everything is running we can configure our environment to allow exploration:\nFirst, we'll need to add the following entries in our `/etc/hosts` file.\n\n```\n127.0.0.1       ca.smallstep.com\n127.0.0.1       internal.smallstep.com\n127.0.0.1       hello.smallstep.com\n```\n\nNow we bootstrap a step certificates environment in a temporary STEPPATH so\nwe won't permanently pollute up our local environment:\n\n```sh\n$ export STEPPATH=/tmp\n$ step ca bootstrap --ca-url https://ca.smallstep.com:9000 --fingerprint 154fa6239ba9839f50b6a17f71addb77e4c478db116a2fbb08256faa786245f5\nThe root certificate has been saved in /tmp/certs/root_ca.crt.\nYour configuration has been saved in /tmp/config/defaults.json.\n```\n\nNow we can use curl to connect. If we don't specify the root certificate we\nwill get the following well-known error:\n\n```sh\n$ curl https://hello.smallstep.com:10000\ncurl: (60) SSL certificate problem: unable to get local issuer certificate\nMore details here: https://curl.haxx.se/docs/sslcerts.html\n\ncurl performs SSL certificate verification by default, using a \"bundle\"\n of Certificate Authority (CA) public keys (CA certs). If the default\n bundle file isn't adequate, you can specify an alternate file\n using the --cacert option.\nIf this HTTPS server uses a certificate signed by a CA represented in\n the bundle, the certificate verification probably failed due to a\n problem with the certificate (it might be expired, or the name might\n not match the domain name in the URL).\nIf you'd like to turn off curl's verification of the certificate, use\n the -k (or --insecure) option.\nHTTPS-proxy has similar options --proxy-cacert and --proxy-insecure.\n```\n\nPassing the `--cacert /tmp/certs/root_ca.crt` flag will make it work as\nexpected, and we'll get a response from the `frontend` server:\n\n```sh\n$ curl --cacert /tmp/certs/root_ca.crt https://hello.smallstep.com:10000\nHello TLS!\n```\n\nTrying the same with the `backend` server we will result in an error because\na mutual TLS connection is required:\n\n```sh\n$ curl --cacert /tmp/certs/root_ca.crt https://internal.smallstep.com:10001\ncurl: (35) error:1401E410:SSL routines:CONNECT_CR_FINISHED:sslv3 alert handshake failure\n```\n\nWe will need to get a client certificate from our internal CA:\n\n```sh\n$ step ca certificate client.smallstep.com client.crt client.key\n✔ Key ID: oA1x2nV3yClaf2kQdPOJ_LEzTGw5ow4r2A5SWl3MfMg (sds@smallstep.com)\n✔ Please enter the password to decrypt the provisioner key: password\n✔ CA: https://ca.smallstep.com:9000\n✔ Certificate: client.crt\n✔ Private Key: client.key\n```\n\nNow trying curl again with both root \u0026 client (we've just generated)\ncertificates, we will get a successful response from the `backend` server:\n\n```sh\n$ curl --cacert /tmp/certs/root_ca.crt --cert client.crt --key client.key https://internal.smallstep.com:10001\nHello mTLS!\n```\n\nThis docker-compose example also includes a SDS server configuration using UNIX\ndomain sockets. Without further modifications we can run the same test sequence\nagainst a different set of ports:\n\n```sh\n$ curl --cacert /tmp/certs/root_ca.crt https://hello.smallstep.com:10010\nHello TLS!\n$ curl --cacert /tmp/certs/root_ca.crt https://internal.smallstep.com:10011\ncurl: (35) error:1401E410:SSL routines:CONNECT_CR_FINISHED:sslv3 alert handshake failure\n$ curl --cacert /tmp/certs/root_ca.crt --cert client.crt --key client.key https://internal.smallstep.com:10011\nHello mTLS!\n```\n\n# Emojivoto example\n\nThe [examples/emojivoto](examples/emojivoto) directory contains an example of\nusing Envoy, [step-sds](https://github.com/smallstep/step-sds) and\n[step certificates](https://github.com/smallstep/certificates) on a simple\nmicroservice application that allows users to vote for their favorite emoji,\nand tracks votes received on a leaderboard. This example uses Buoyant's\n[emojivoto](https://github.com/BuoyantIO/emojivoto) as its basis.\n\nThe application is composed of the following 3 services:\n\n* `emojivoto-web`: Web frontend and REST API\n* `emojivoto-emoji-svc`: gRPC API for finding and listing emoji\n* `emojivoto-voting-svc`: gRPC API for voting and leaderboard\n\nBesides using gRPC, the application does not come with mutual TLS support out\nof the box. We will use Envoy and step-sds as a highly simplified service mesh\nthat will handle the communications between services using mutual TLS.\n\nIn our example, all the services will be behind an ingress proxy and a TLS\ncertificate will be available for all of them. Both gRPC services will require a\nclient certificate from our internal Certificate Authority, so only mTLS connections\nwill be allowed. The web service that is the one connecting to the gRPC services\nwill use an egress proxy in Envoy with a client certificate, so it will be able to\nconnect to it.\n\nThe emojivoto example uses kubernetes, so you will need to have access to a\nkubernetes cluster, if you don't\n[minikube](https://kubernetes.io/docs/tasks/tools/install-minikube/) or\n[docker](https://www.docker.com) provides you with options.\n\nRun the following commands to set up this emojivoto example:\n\n```sh\n$ cd examples/emojivoto\n$ make\nkubectl apply -f ca.yaml\nnamespace/step created\nsecret/step-certificates-ca-password created\nsecret/step-certificates-provisioner-password created\nconfigmap/step-certificates-config created\nconfigmap/step-certificates-certs created\nconfigmap/step-certificates-secrets created\nservice/ca created\ndeployment.apps/step-certificates created\nsleep 2\nkubectl -n step wait --for=condition=Ready -l app.kubernetes.io/name=step-certificates pod\npod/step-certificates-6fc86d5689-spzvv condition met\nkubectl apply -f emojivoto.yaml\nnamespace/emojivoto created\nserviceaccount/emoji created\nserviceaccount/voting created\nserviceaccount/web created\nsecret/step-sds-secrets created\nconfigmap/step-sds-certs created\nconfigmap/step-sds-config created\nconfigmap/envoy-web-config created\nconfigmap/envoy-emoji-config created\nconfigmap/envoy-voting-config created\ndeployment.apps/emoji created\nservice/emoji-svc created\ndeployment.apps/voting created\nservice/voting-svc created\ndeployment.apps/web created\nservice/web-svc created\n```\n\nThis will install [step certificates](https://github.com/smallstep/certificates) as a\nonline Certificate Authority in the step namespace and the emojivoto services in the\nnamespace with the same name. To test it locally you will need to edit your `/etc/hosts`\nfile and point `web-svc.emojivoto` to the ClusterIP of the `web-svc` service, and then\njust go to `https://web-svc.emojivoto`. Here's how you retrieve the ClusterIP:\n\n```sh\n$ kubectl get service -n emojivoto web-svc\nNAME      TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE\nweb-svc   ClusterIP   10.59.249.130   \u003cnone\u003e        443/TCP   1h\n```\n\nIn case `web-svc`'s ClusterIP is not route-able (running inside AWS or GCP) you can use `kubectl`\nport forwarding instead. Make sure to point your `/etc/hosts` entry for `web-svc.emojivoto`\nat `127.0.0.1` and run following command:\n\n```sh\n$ kubectl port-forward -n emojivoto service/web-svc --address 127.0.0.1 7443:443\n```\n\nThe certificate of our web app is signed by our internal CA and you will see the unsafe\nwarning in your browser as its not included in local trust stores. If you want to avoid\nthe warning message you can always install the root certificate into your trust store:\n\n```sh\n$ cat \u003c\u003cEOF \u003e /tmp/root_ca.crt\n-----BEGIN CERTIFICATE-----\nMIIBhTCCASugAwIBAgIQTiiy0M/WWuVz2cDakLykdzAKBggqhkjOPQQDAjAhMR8w\nHQYDVQQDExZTbWFsbHN0ZXAgVGVzdCBSb290IENBMB4XDTE5MDcxMjIyMTQxNFoX\nDTI5MDcwOTIyMTQxNFowITEfMB0GA1UEAxMWU21hbGxzdGVwIFRlc3QgUm9vdCBD\nQTBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABNsTsgcRwTakVB+ouxeWzefBaLxu\nhq/7d4qLbGw5pGixG0f6kN4HtIVxjZru+ABRL3PjKWUffXWiJD8XK2/QJSmjRTBD\nMA4GA1UdDwEB/wQEAwIBBjASBgNVHRMBAf8ECDAGAQH/AgEBMB0GA1UdDgQWBBSF\nidiUKAm0h3qnuYHq4MqgpzZsODAKBggqhkjOPQQDAgNIADBFAiEAwwKqV1AxH4ss\nU69xQ6ZYIjv6l7xLWkFwDaZQXFtLsyYCIBuUpyIHlZBA0Vp5TPZgdiXIpcIrr8+z\n5bpQRw86QnPY\n-----END CERTIFICATE-----\nEOF\n$ step certificate install /tmp/root_ca.crt\nCertificate /tmp/root_ca.crt has been installed.\nX.509v3 Root CA Certificate (ECDSA P-256) [Serial: 1038...4951]\n  Subject:     Smallstep Test Root CA\n  Issuer:      Smallstep Test Root CA\n  Valid from:  2019-07-12T22:14:14Z\n          to:  2029-07-09T22:14:14Z\n```\n\nRemember to remove the root certificate from your local trust store after\nlocal testing as this certificate is public (as part of this repo) and\nanyone can use it:\n\n```sh\n$ step certificate uninstall /tmp/root_ca.crt\nCertificate /tmp/root_ca.crt has been removed.\nX.509v3 Root CA Certificate (ECDSA P-256) [Serial: 1038...4951]\n  Subject:     Smallstep Test Root CA\n  Issuer:      Smallstep Test Root CA\n  Valid from:  2019-07-12T22:14:14Z\n          to:  2029-07-09T22:14:14Z\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsmallstep%2Fstep-sds","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsmallstep%2Fstep-sds","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsmallstep%2Fstep-sds/lists"}