Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/aatarasoff/spring-cloud-marathon

Spring Cloud integration with Mesos and Marathon
https://github.com/aatarasoff/spring-cloud-marathon

marathon mesos ribbon service-discovery spring-boot spring-boot-starter spring-cloud

Last synced: 29 days ago
JSON representation

Spring Cloud integration with Mesos and Marathon

Awesome Lists containing this project

README 

        
# Spring Cloud Marathon



[![Join the chat at https://gitter.im/aatarasoff/spring-cloud-marathon](https://badges.gitter.im/aatarasoff/spring-cloud-marathon.svg)](https://gitter.im/aatarasoff/spring-cloud-marathon?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) [![Build Status](https://travis-ci.org/aatarasoff/spring-cloud-marathon.svg?branch=master)](https://travis-ci.org/aatarasoff/spring-cloud-marathon) [![Coverage Status](https://coveralls.io/repos/github/aatarasoff/spring-cloud-marathon/badge.svg?branch=master)](https://coveralls.io/github/aatarasoff/spring-cloud-marathon?branch=master)



This project helps with integration between [Spring Cloud](http://projects.spring.io/spring-cloud/) and [Marathon framework](https://mesosphere.github.io/marathon/) for [Apache Mesos](http://mesos.apache.org/)



## How to connect the project



Add `jcenter` repository:

```groovy

repositories {

    jcenter()

}

```



or for maven:

```xml



    

        jcenter

        http://jcenter.bintray.com/

    



```



And add dependency with latest version (or feel free to choose specific)

```groovy

compile 'info.developerblog.spring.cloud:spring-cloud-marathon-starter:+'

```



or for maven:

```xml



    info.developerblog.spring.cloud

    spring-cloud-marathon-starter

    x.y.z



```



## Motivation



Mesos and Marathon helps you to orchestrate microservices or other artifacts in distributed systems. In fact Marathon keeps information about current configuration including location of service, its healhchecks etc. So, do we need third-party system that keeps configuration and provides service discovery features? If you don't have any serious reason the answer will be "No".



## Supported Spring Cloud patterns



`DiscoveryClient` implementation (supports Ribbon, Feign and Zuul).



`Client Load Balancing` with Ribbon and Hystrix.



Other distributed systems patterns from Spring Cloud.



## Marathon configuration



All configuration should be provided in `bootstrap.yml`



### Development mode (without Marathon)



```yaml

spring.cloud.marathon.enabled: false

```



### Standalone mode (single-host Marathon)



```yaml

spring:

    cloud:

        marathon:

            scheme: http       #url scheme

            host: marathon     #marathon host

            port: 8080         #marathon port

```



### Production mode



Authentication via Marathon, providing a list of marathon masters:

```yaml

spring:

    cloud:

        marathon:

            listOfServers: m1:8080,m2:8080,m3:8080       #list of marathon masters

            token:                       #DC/OS HTTP API Token (optional)

            username: marathon                           #username for basic auth (optional)

            password: mesos                              #password for basic auth (optional)

```



or, provide a load balanced marathon endpoint:

```yaml

spring:

    cloud:

        marathon:

            endpoint: http://marathon.local:8080         #override scheme+host+port

            token:                       #DC/OS HTTP API Token (optional)

            username: marathon                           #username for basic auth (optional)

            password: mesos                              #password for basic auth (optional)

```



Other configuration for services and discovery you can see in [official documentation](http://cloud.spring.io/spring-cloud-static/Camden.SR3/)



### Authentication



Spring Cloud Marathon supports four methods of authentication:



- No Authentication (using Marathon Endpoint)

- Basic Authentication (using Marathon Endpoint)

- Token Authentication (using Marathon Endpoint)



#### No Authentication



Do not specify username, password, token or dcosPrivateKey

Specify a load balanced endpoint or listOfServers that describe the Marathon Endpoint. e.g.



```yaml

spring:

    cloud:

        marathon:

            listOfServers: m1:8080,m2:8080,m3:8080      #list of marathon masters

```



#### Basic Authentication



Provide a username & password.  Specify an endpoint or listOfServers that describe the Marathon Endpoint. e.g.



```yaml

spring:

    cloud:

        marathon:

            listOfServers: m1:8080,m2:8080,m3:8080       #list of marathon masters

            username: marathon                           #username for basic auth (optional)

            password: mesos                              #password for basic auth (optional)

```



#### Token Authentication



Provide a HTTP API token (note: tokens expire after 5 days).  Specify a load balanced endpoint or listOfServers that describe the Marathon Endpoint. e.g.



```yaml

spring:

    cloud:

        marathon:

            listOfServers: m1:8080,m2:8080,m3:8080       #list of marathon masters

            token:                       #DC/OS HTTP API Token (optional)

```



### Services configuration



There is one specific moment for services notation and their configuration. In Marathon service id has following pattern:

```text

/group/path/app

```



and symbol `/` is not allowed as a virtual host in Feign or RestTemplate. So we cannot use original service id as Spring Cloud service id. Instead of `/` in this implementation other separator: `.` is used. That means that service with id: `/group/path/app` has internal presentation: `group.path.app`.



And you should configure them like:

```yaml

group.path.app:

    ribbon:

        

```



If a specific service cannot be located by the id, then a second lookup is performed for services that contain the given id. e.g.

A service has been deployed using three different Marathon service ids:

```text

/group1/path/app

/group2/path/app

/group3/path/app

```

A client is configured for the service name only, excluding the group & path from the id:

```yaml

app:

    ribbon:

        

```

Service Tasks for all three services will be discovered & used by ribbon.



Sometimes it is useful discover services that are advertising a specific capability; by API version for example:

Three versions of a service have been deployed, with the following Marathon service ids and labels:

```text

/group1/path/app  "labels":{ "API_VERSION" : "V1" }

/group2/path/app  "labels":{ "API_VERSION" : "V2" }

/group3/path/app  "labels":{ "API_VERSION" : "V2" }

```

A client is configured to expect the "V2" API Version:

```yaml

app:

    ribbon:

        

        MetaDataFilter:

            API_VERSION: V2

```

Only service instances that contain "app" in the service id and have matching labels will be discovered & used by ribbon.



Where multiple values are specified for MetaDataFilter, all values must match service labels before ribbon will use the service instance:

```yaml

app:

    ribbon:

        

        MetaDataFilter:

            API_VERSION: V2

            ENVIRONMENT: UAT

            APPLICATION_OWNER: [email protected]

```



Combining the loose service id matching with service label filtering permits us to get creative with service discovery:

```yaml

group.path:

    ribbon:

        

        MetaDataFilter:

            CUSTOMER_ENTITY: V1

```

i.e. select any service deployed to /group/path that supports Version 1 of the Customer Entity



Ignore the service id entirely and only match using service labels:

```yaml

customer:

    ribbon:

      

      IgnoreServiceId: true

      MetaDataFilter:

        CUSTOMER_ENTITY: V1

        ENVIRONMENT: UAT

```

i.e. select any service in the UAT environment that supports Version 1 of the Customer Entity.  

Note: The service id of 'customer' is ignored, so all services will be filtered by service label



Specify equality based selector on service labels

```yaml

customer:

    ribbon:

    

    MetaDataFilter:

      API_VERSION: '!=V3'  # not equal to V3; note: must be quoted to keep yaml happy

      ENVIRONMENT: '==DEV' # equal to DEV; note: must be quoted to keep yaml happy

      CUSTOMER_ENTITY: V1  # equal to V1; default is equals

```



Specify set based selector on service labels

```yaml

customer:

    ribbon:

    

    MetaDataFilter:

      API_VERSION: in(V1,V2)       # is V1 OR is V2

      ENVIRONMENT: notin(UAT,PROD) # is not UAT AND is not PROD

```



#### Zones

Zones support is provided by fetching them from instance's hostname. For example, mesos slaves have hostname like following:

```text

slave1.dc1

slave2.dc1

slave1.dc2

...

```

There is regexp pattern `.+\.(.+)` for fetching zone (or datacenter) exists. So, you could define it in service configuration:

```yaml

customer:

    ribbon:

      

      ZonePattern: '.+\.(.+)'

```

Finally, you should define zone for app:

```yaml

spring.cloud.marathon.discovery.zone: dc1

```

And then different zone-aware filters and rules will be applied.



## Running the example



Build sample application docker image:

```bash

./gradlew dockerBuild

```



Install native docker on Linux or docker for MacOS X or Windows and run `docker-compose` for local environment deployment with zookeeper, mesos and marathon:

```bash

docker-compose up -d

```



Add following record into your `/etc/hosts` file:

```bash

127.0.0.1 mesos-slave.dc1

127.0.0.1 mesos-slave.dc2

```



Then upload `test-marathon-app-manifest.json` as application manifest:

```bash

curl -XPOST http://:8080/v2/apps?force=true -H "Content-Type: application/json" --data-binary @test-marathon-app-manifest.json -v

```



and run the example application:

```bash

./gradlew bootRun

```



Now you may test application by curl:

```bash

curl localhost:9090/instances

curl localhost:9090/feign

```



## Enjoy!