Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/heroku/heroku-kong

🐒 Kong API gateway as a Heroku app
https://github.com/heroku/heroku-kong

Last synced: 2 months ago
JSON representation

🐒 Kong API gateway as a Heroku app

Awesome Lists containing this project

README 

        
Kong as a Heroku app

====================



⚠️ **This project is no longer maintained.**



Deploy [Kong 1.0](https://konghq.com/blog/kong-1-0-ga/) clusters to Heroku Common Runtime and Private Spaces using the [Kong buildpack](https://github.com/heroku/heroku-buildpack-kong/).



⏫ **Upgrading from an earlier version?** See [Upgrade Guide](#user-content-upgrade-guide).



🔬 This is a community proof-of-concept, [MIT license](LICENSE), provided "as is", without warranty of any kind.



[![Deploy to Heroku](https://www.herokucdn.com/deploy/button.svg)](https://heroku.com/deploy)



* [Purpose](#user-content-purpose)

* [Usage](#user-content-usage)

  * [Deploy](#user-content-deploy)

  * [Connect local to Heroku app](#user-content-connect-local-to-heroku-app)

  * [Admin console](#user-content-admin-console)

  * [Admin API](#user-content-admin-api)

    * [The API Key](#user-content-admin-api-key)

    * [Accessing](#user-content-accessing-the-external-admin-api)

    * [Disabling](#user-content-disabling-the-external-admin-api)

  * [Terraform](#user-content-terraform)

  * [Upgrade guide](#user-content-upgrade-guide)

* [Customization](#user-content-customization)

  * [Configuration](#user-content-configuration)

  * [Kong plugins & additional Lua modules](#user-content-kong-plugins--additional-lua-modules)

* [Demos](#user-content-demos)

  * [API Rate Limiting](#user-content-demo-api-rate-limiting)

  * [Custom plugin: hello-world-header](#user-content-demo-custom-plugin-hello-world-header)

  * [Custom plugin: API translation, JSON→XML](#user-content-demo-custom-plugin-api-translation-jsonxml)

* [Development Notes](#user-content-dev-notes)

  * [Learning the Language of Kong](#user-content-learning-the-language-of-kong)

  * [Programming with Lua](#user-content-programming-with-lua)

  * [Local Development](#user-content-local-development)

    * [Requirements](#user-content-requirements)

    * [Clone & connect](#user-content-clone--connect)

    * [Setup](#user-content-setup)

    * [Running](#user-content-running)

    * [Testing](#user-content-testing)



Purpose

-------

Kong is an extensible [web proxy](https://en.m.wikipedia.org/wiki/Proxy_server) based on [OpenResty](http://openresty.org/en/), a web app framework built on the embedded [Lua language](http://www.lua.org) capabilities of the [Nginx web server](http://nginx.org/en/).



With Heroku, Kong may be used for a variety of purposes. A few examples:



  * unify access control & observability for a suite of microservices

  * enforce request rate & size limits globally, based on the endpoint, or the authenticated consumer

  * create a single management point for routing requests based on DNS hostnames, URL paths, and HTTP headers



🦍 Visit [Kong HQ](https://konghq.com), the official resource for everything Kong.



Usage

-----



### Deploy



Use the deploy button to create a Kong app in your Heroku account:



[![Deploy to Heroku](https://www.herokucdn.com/deploy/button.svg)](https://heroku.com/deploy)



### Connect local to Heroku app



To make changes to the Kong app's source, clone and connect this repo (or your own fork) to the Heroku app:



```bash

git clone https://github.com/heroku/heroku-kong.git

cd heroku-kong



# Use the name of the Heroku app:

heroku git:remote --app $APP_NAME

heroku info

```



### Admin console



To gain local console access to Kong deployed on Heroku, see [ADMIN](ADMIN.md).



Console access is primarily useful for performing `kong` CLI commands against the deployed app. Most administrative features do not require console access and instead are available through the Kong Admin API.



### Admin API



When this app is deployed to Heroku, it automatically provisions a protected, external-facing [loopback proxy](https://docs.konghq.com/1.0.x/secure-admin-api/#kong-api-loopback) to [Kong's Admin API](https://docs.konghq.com/1.0.x/admin-api/), secured by the `KONG_HEROKU_ADMIN_KEY` config var.



#### Admin API key



`KONG_HEROKU_ADMIN_KEY` is generated automatically when this app is [deployed using the automated app setup](#user-content-deploy).



You can explicitly set a new admin key value:



```bash

heroku config:set KONG_HEROKU_ADMIN_KEY=xxxxx

```



⚠️ **Always set a unique, cryptographically strong key value.** A weak admin key may result in the proxy being compromised and abused by malicious actors.



#### Accessing the external Admin API



Make HTTPS requests using a tool like [`curl`](https://curl.haxx.se) or [Paw.cloud](https://paw.cloud):



1. Base URL of the app's [Kong Admin API](https://docs.konghq.com/1.0.x/admin-api/) is `https://$APP_NAME.herokuapp.com/kong-admin`

2. Set the current [admin key](#user-content-admin-api-key) in the `apikey` HTTP header



For example, set the current admin key into a local shell variable:



```bash

KONG_HEROKU_ADMIN_KEY=`heroku config:get KONG_HEROKU_ADMIN_KEY`

```



Now use the following HTTP request style to interact with the [Kong's Admin API](https://docs.konghq.com/1.0.x/admin-api/):



✏️ *Replace the variable `$APP_NAME` with value for your unique deployment.*



```bash

curl -H "apikey: $KONG_HEROKU_ADMIN_KEY" https://$APP_NAME.herokuapp.com/kong-admin/status

```



#### Disabling the external Admin API



If you prefer to only use the [console-based Admin API](ADMIN.md), then this externally-facing proxy can be disabled:



```bash

curl -H "apikey: $KONG_HEROKU_ADMIN_KEY" https://$APP_NAME.herokuapp.com/kong-admin/services/kong-admin/routes

# For the returned Route's `id`,

curl -H "apikey: $KONG_HEROKU_ADMIN_KEY" -X DELETE https://$APP_NAME.herokuapp.com/kong-admin/routes/$ROUTE_ID

# Now there's no longer admin access!

# Finally, clear out the old admin key value.

heroku config:unset KONG_HEROKU_ADMIN_KEY

```



### Terraform



Kong may be provisioned and configured on Heroku using [Hashicorp Terraform](https://www.terraform.io) and a third-party [Kong provider](https://github.com/kevholditch/terraform-provider-kong).



See these examples of [Using Terraform with Heroku](https://devcenter.heroku.com/articles/using-terraform-with-heroku):



* [Common Runtime microservices with a unified gateway](https://github.com/mars/terraform-heroku-common-kong-microservices)

* [Private Spaces microservices with a unified gateway](https://github.com/mars/terraform-heroku-enterprise-kong-microservices)



### Upgrade guide



🚨 **Potentially breaking changes.** Please attempt upgrades on a staging system before upgrading production.



#### The buildpack



[Buildpack v6.0.0](https://github.com/heroku/heroku-buildpack-kong/releases) supports rapid deployments using a

pre-compiled Kong binary. A pre-existing, customized app may require changes continue functioning, if the app explicitly uses the `/app/.heroku` directory prefix.



[Buildpack v7.0.0-rc\*](https://github.com/heroku/heroku-buildpack-kong/releases) supports Kong 1.0 release candidates. The "rc" releases only support upgrading from Kong 0.14, not earlier versions or other release candidates.



[Buildpack v7.0.0](https://github.com/heroku/heroku-buildpack-kong/releases) supports Kong 1.0.



▶️ See [UPGRADING the buildpack](https://github.com/heroku/heroku-buildpack-kong/blob/master/UPGRADING.md).



#### Kong



First, see [Kong's official upgrade path](https://github.com/Kong/kong/blob/master/UPGRADE.md).



Then, take into account these facts about how this Kong on Heroku app works:



* this app automatically runs `kong migrations up` for every deployment

* you may prevent the previous version of Kong from attempting to use the new database schema during the upgrade (this will cause downtime):

   1. check the current formation size with `heroku ps`

   1. scale the web workers down `heroku ps:scale web=0`

   1. [perform the upgrade](https://github.com/Kong/kong/blob/master/UPGRADE.md)

   1. allow release process to run

   1. finally restart to the original formation size `heroku ps:scale web=$PREVIOUS_SIZE`

* once Kong 1.0 is successfully deployed, execute: `

heroku run "kong migrations finish --conf /app/config/kong.conf"`



🏥 Please [open an issue](https://github.com/heroku/heroku-kong/issues), if you encounter problems or have feedback about this process.



Customization

-------------

Kong may be customized through configuration and plugins.



### Configuration



Kong is automatically configured at runtime with a `.profile.d` script:



  * renders the `config/kong.conf` file based on:

    * the customizable [`config/kong.conf.etlua`](config/kong.conf.etlua) template

    * values of [config vars, as defined in the buildpack](https://github.com/heroku/heroku-buildpack-kong#user-content-environment-variables)

  * exports environment variables

    * see: `.profile.d/kong-env` in a running dyno



All file-based config may be overridden by setting `KONG_`-prefixed config vars, e.g. `heroku config:set KONG_LOG_LEVEL=debug`



👓 See: [Kong 1.0 Configuration Reference](https://docs.konghq.com/1.0.x/configuration/)



### Kong plugins & additional Lua modules



👓 See: [buildpack usage](https://github.com/heroku/heroku-buildpack-kong#user-content-usage)



Demos

-----

Usage examples and sample plugins are includes with this Heroku Kong app.



### Demo: [API Rate Limiting](https://getkong.org/plugins/rate-limiting/)



Request [this Bay Lights API](https://kong-proxy-public.herokuapp.com/bay-lights/lights) more than five times in a minute, and you'll get **HTTP Status 429: API rate limit exceeded**, along with `X-Ratelimit-Limit-Minute` & `X-Ratelimit-Remaining-Minute` headers to help the API consumers regulate their usage.



Try it in your shell terminal:

```bash

curl --head https://kong-proxy-public.herokuapp.com/bay-lights/lights

# HTTP/1.1 200 OK

curl --head https://kong-proxy-public.herokuapp.com/bay-lights/lights

# HTTP/1.1 200 OK

curl --head https://kong-proxy-public.herokuapp.com/bay-lights/lights

# HTTP/1.1 200 OK

curl --head https://kong-proxy-public.herokuapp.com/bay-lights/lights

# HTTP/1.1 200 OK

curl --head https://kong-proxy-public.herokuapp.com/bay-lights/lights

# HTTP/1.1 200 OK

curl --head https://kong-proxy-public.herokuapp.com/bay-lights/lights

# HTTP/1.1 429

```



Here's the whole configuration for this API rate limiter:



```bash

curl http://localhost:8001/services/ -i -X POST \

  --data 'name=bay-lights' \

  --data 'protocol=https' \

  --data 'port=443' \

  --data 'host=bay-lights-api-production.herokuapp.com'

# Note the Service ID returned in previous response, use it in place of `$SERVICE_ID`.

curl http://localhost:8001/plugins/ -i -X POST \

  --data 'name=request-size-limiting' \

  --data "config.allowed_payload_size=8" \

  --data "service.id=$SERVICE_ID"

curl http://localhost:8001/plugins/ -i -X POST \

  --data 'name=rate-limiting' \

  --data "config.minute=5" \

  --data "service.id=$SERVICE_ID"

curl http://localhost:8001/routes/ -i -X POST \

  --data 'paths[]=/bay-lights' \

  --data "service.id=$SERVICE_ID"

```



### Demo: custom plugin: hello-world-header



[Custom plugins](https://docs.konghq.com/1.0.x/plugin-development/) allow you to observe and transform HTTP traffic using lightweight, high-performance [Lua](http://www.lua.org) code in Nginx [request processing contexts](https://docs.konghq.com/1.0.x/plugin-development/custom-logic/#available-request-contexts). Building on the [previous example](#user-content-demo-api-rate-limiting), let's add a simple plugin to Kong.



[hello-world-header](lib/kong/plugins/hello-world-header/handler.lua) will add an HTTP response header **X-Hello-World** showing the date and a message from an environment variable.



Activate this plugin for the API:



```bash

curl http://localhost:8001/plugins/ -i -X POST \

  --data 'name=hello-world-header' \

  --data "service.id=$SERVICE_ID"

```



Then, set a message through the Heroku config var:



```bash

heroku config:set HELLO_WORLD_MESSAGE='🌈🙈'

# …the app will restart.

```



Now, when fetching an API response, notice the **X-Hello-World** header:



```bash

curl --head https://kong-proxy-public.herokuapp.com/bay-lights/lights

# ↩︎

# HTTP/1.1 200 OK

# Connection: keep-alive

# Content-Type: application/json;charset=utf-8

# Content-Length: 9204

# X-Ratelimit-Limit-Minute: 5

# X-Ratelimit-Remaining-Minute: 4

# Server: Cowboy

# Date: Mon, 28 Aug 2017 23:14:47 GMT

# Strict-Transport-Security: max-age=31536000

# X-Content-Type-Options: nosniff

# Vary: Accept-Encoding

# Request-Id: 6c815aae-a5e9-496f-b731-dc72bbe2b63e

# Via: kong/0.14.0, 1.1 vegur

# X-Hello-World: Today is 2017-08-28. 🌈🙈  <--- The injected header

# X-Kong-Upstream-Latency: 49

# X-Kong-Proxy-Latency: 161

```



### Demo: custom plugin: API translation, JSON→XML



JSON/REST has taken over as the internet API lingua franca, shedding the complexity of XML/SOAP. The [National Digital Forecast Database [NDFD]](http://graphical.weather.gov/xml/) is a legacy XML/SOAP service.



This app includes a sample, custom plugin [ndfd-xml-as-json](lib/kong/plugins/ndfd-xml-as-json). This plugin exposes a JSON API that returns the maximum temperatures forecast for a location from the NDFD SOAP service. Using the single-resource concept of REST, the many variations of a SOAP or other legacy interfaces may be broken out into elegant, individual JSON APIs.



Try it in your shell terminal:

```bash

curl https://kong-proxy-public.herokuapp.com/ndfd-max-temps \

  --data '{"latitude":37.733795,"longitude":-122.446747}'

# Response contains max temperatures forecast for San Francisco, CA



curl https://kong-proxy-public.herokuapp.com/ndfd-max-temps \

 --data '{"latitude":27.964157,"longitude":-82.452606}'

# Response contains max temperatures forecast for Tampa, FL



curl https://kong-proxy-public.herokuapp.com/ndfd-max-temps \

  --data '{"latitude":41.696629,"longitude":-71.149994}'

# Response contains max temperatures forecast for Fall River, MA

```



Much more elegant than the legacy API. See the [sample request body](spec/data/ndfd-request.xml):

```bash

curl --data @spec/data/ndfd-request.xml -H 'Content-Type:text/xml' -X POST https://graphical.weather.gov/xml/SOAP_server/ndfdXMLserver.php

# Response contains wrapped XML data. Enjoy decoding that.

```



This technique may be used to create a suite of cohesive JSON APIs out of various legacy APIs.



Here's the configuration for this API translator:



```bash

curl http://localhost:8001/services/ -i -X POST \

  --data 'name=ndfd-max-temps' \

  --data 'protocol=https' \

  --data 'port=443' \

  --data 'host=graphical.weather.gov' \

  --data 'path=/xml/SOAP_server/ndfdXMLserver.php'

# Note the Service ID returned in previous response, use it in place of `$SERVICE_ID`.

curl http://localhost:8001/plugins/ -i -X POST \

  --data 'name=request-size-limiting' \

  --data "config.allowed_payload_size=8" \

  --data "service.id=$SERVICE_ID"

curl http://localhost:8001/plugins/ -i -X POST \

  --data 'name=rate-limiting' \

  --data "config.minute=5" \

  --data "service.id=$SERVICE_ID"

curl http://localhost:8001/plugins/ -i -X POST \

  --data 'name=ndfd-xml-as-json' \

  --data "service.id=$SERVICE_ID"

curl http://localhost:8001/routes/ -i -X POST \

  --data 'paths[]=/ndfd-max-temps' \

  --data "service.id=$SERVICE_ID"

```



👓 See the implementation of the custom plugin's [Lua source code](lib/kong/plugins/ndfd-xml-as-json), [unit tests](spec/unit/kong/plugins/ndfd-xml-as-json/handler_spec.lua), and [integration tests](spec/integration/kong/plugins/ndfd-xml-as-json_spec.lua).



## Dev notes



### Learning the language of Kong



* [Definitely an openresty guide](http://www.staticshin.com/programming/definitely-an-open-resty-guide/)

* [An Introduction To OpenResty - Part 1](http://openmymind.net/An-Introduction-To-OpenResty-Nginx-Lua/), [2](http://openmymind.net/An-Introduction-To-OpenResty-Part-2/), & [3](http://openmymind.net/An-Introduction-To-OpenResty-Part-3/)

* [Nginx API for Lua](https://github.com/openresty/lua-nginx-module#nginx-api-for-lua), `ngx` reference, for use in Kong plugins

  * [Nginx variables](http://nginx.org/en/docs/http/ngx_http_upstream_module.html#var_upstream_status), accessible through `ngx.var`



### Programming with Lua



* [Lua 5.1](http://www.lua.org/manual/5.1/), Note: Kong is not compatible with the newest Lua version

* [Classic Objects](https://github.com/rxi/classic), the basis of Kong's plugins

* [Moses](http://yonaba.github.io/Moses/doc/), functional programming

* [Lubyk](http://doc.lubyk.org), realtime programming (performance- & game-oriented)

* [resty-http](https://github.com/pintsized/lua-resty-http), Nginx-Lua co-routine based HTTP client

* [Serpent](http://notebook.kulchenko.com/programming/serpent-lua-serializer-pretty-printer), inspect values

* [Busted](http://olivinelabs.com/busted/), testing framework



### Local development



To work with Kong locally on macOS X.



#### Requirements



* [git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)

* [Heroku](https://www.heroku.com/home)

  * [command-line tools (CLI)](https://toolbelt.heroku.com)

  * [a free account](https://signup.heroku.com)

* Kong & its dependencies

  * as a single package install

    * [Kong](https://docs.konghq.com/install/macos/) for macOS

  * or, from source

    * [Lua](https://www.lua.org/versions.html) 5.1

    * [LuaRocks](https://github.com/luarocks/luarocks) 2.4.4

    * [OpenSSL](https://www.openssl.org/source/) 1.1.1

    * [OpenResty](https://openresty.org/en/installation.html) 1.13.6.2

      * [Install from source](https://docs.konghq.com/install/source/)

      * `./configure -j2 --with-openssl=~/Downloads/openssl-1.1.1a --with-http_realip_module --with-http_stub_status_module`

    * [Kong](https://github.com/Kong/kong) 1.0.0

      * `luarocks install kong 1.0.0 OPENSSL_DIR=/usr/local/opt/openssl CRYPTO_DIR=/usr/local/opt/openssl`



#### Clone & connect



If you haven't already, clone and connect your own fork of this repo to the Heroku app:



```bash

# Replace the main repo with your own fork:

git clone https://github.com/heroku/heroku-kong.git

cd heroku-kong



# Use the name of the Heroku app:

heroku git:remote --app $APP_NAME

heroku info

```



##### Setup



1. Ensure [requirements](#user-content-requirements) are met

1. Create the Postgres user & databases:



    ```bash

    createuser --pwprompt kong

    # set the password "kong"



    createdb --owner=kong kong_dev

    createdb --owner=kong kong_tests

    ```

1. Execute `./bin/setup`



##### Running



```bash

bin/start

```



* Logs in `/usr/local/var/kong/logs/`

* Prefix is `/usr/local/var/kong` for commands like:

  * `kong health -p /usr/local/var/kong`

  * `kong stop -p /usr/local/var/kong`



##### Testing



Any test-specific Lua rocks should be specified in `Rockfile_test` file, so that they are not installed when the app is deployed.



Add tests in `spec/`:



  * Uses the [Busted testing framework](http://olivinelabs.com/busted)

  * [Kong plugin testing guide](https://docs.konghq.com/1.0.x/plugin-development/tests/)

  * [buildpack requirements for testing](https://github.com/heroku/heroku-buildpack-kong/blob/master/README.markdown#user-content-testing)



```bash

bin/test

```