Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/lyft/discovery

This service provides a REST interface for querying for the list of hosts that belong to all microservices.
https://github.com/lyft/discovery

lyft

Last synced: 2 months ago
JSON representation

This service provides a REST interface for querying for the list of hosts that belong to all microservices.

Host: GitHub
URL: https://github.com/lyft/discovery
Owner: lyft
License: other
Archived: true
Created: 2016-08-12T18:34:59.000Z (over 8 years ago)
Default Branch: master
Last Pushed: 2024-04-30T20:59:28.000Z (9 months ago)
Last Synced: 2024-08-02T01:27:35.627Z (6 months ago)
Topics: lyft
Language: Python
Homepage:
Size: 56.6 KB
Stars: 184
Watchers: 140
Forks: 43
Open Issues: 5
Metadata Files:
- Readme: README.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md

Awesome Lists containing this project

awesome-repositories - lyft/discovery - This service provides a REST interface for querying for the list of hosts that belong to all microservices. (Python)

README

⚠️ This repository has been archived and is no longer accepting contributions ⚠️

# Discovery

This service provides a REST interface for querying for the list of hosts that belong to a given service in microservice infrastructure.
Host information is written to and read from backend store (DynamoDB by default). This project relies on the following libraries:
* Flask (web-framework)
* Flask-Cache (for caching and reusing results for GET requests)
* Pynamodb (for reading/writing DynamoDB data)

Also check [requirements.txt](https://github.com/lyft/discovery/blob/master/requirements.txt) for other dependencies.

## Discovery service settings
Discovery service settings are controlled by [environment variables](https://github.com/lyft/discovery/blob/master/app/settings.py).

* HOST_TTL
* If the last heartbeat was not performed in the last HOST_TTL seconds, discovery service will remove host from backend storage. Default value is 600 (10 minutes).
* CACHE_TTL
* Flask cache expiration in seconds, discovery calls BACKEND_STORAGE to fill the cache.
This cache is used for hosts retrieval by [service](#get-v1registrationservice) or [service repo](#get-v1registrationreposervice_repo_name).
Default value is 30 seconds.
* BACKEND_STORAGE
* Type of the backend storage used in discovery service. Supported values are: DynamoDB, InMemory, InFile.
By default DynamoDB backend is used.
* CACHE_TYPE
* Supported values 'simple' or 'null'. Default value is 'null' which effectively turn flask caching off.
* APPLICATION_DIR
* Application directory.
* APPLICATION_ENV
* Environment discovery service runs in, e.g., development or production. Default value is development.
* DEBUG
* If debug mode is used. Default value is true.
* LOG_LEVEL
* Set logging level. Default value is DEBUG.
* PORT
* Port flask app using. Default value is 8080.
* DYNAMODB_TABLE_HOSTS
* Used only in case of DynamoDB backend.
* DYNAMODB_URL
* Used only for development in case of DynamoDB backend running locally.
* DYNAMODB_CREATE_TABLES_IN_APP
* Used for creating DynamoDB table, useful only in case DynamoDB backend storage used.

## API
### GET /v1/registration/:service
Returns metadata for the given `:service`.

* service
* *(required, string)* name of the service metadata is queried for.

On successful response, response body will be in the following JSON format:
```json
{
"env": "...",
"hosts": [],
"service": "..."
}
```
* env
* *(required, string)* environment discovery service runs in, e.g., development or production.
* hosts
* *(required, object)* list of non expired hosts (hosts that last checked in to discovery service in the last HOST_TTL period).
Each host is in the following JSON format:

```json
{
"ip_address": "...",
"last_check_in": "...",
"port": 9211,
"revision": "...",
"service": "...",
"service_repo_name": "...",
"tags": {}
}
```
* ip_address
* *(required, string)* ip address of the host.
* last_check_in
* *(required, string)* heartbeat timestamp converted to string, last time host registered with discovery service.
* port
* *(required, integer)* port on which the host expects connections, Envoy will connect to this port.
* revision
* *(required, string)* service SHA running on the host.
* service
* *(required, string)* service name.
* service_repo_name
* *(required, string)* service repo, used for selecting hosts based on the service_repo (can be empty if not set).
* tags
* *(required, object)* see tags [here](#tags-json).
* service
* *(required, string)* service name.

### GET /v1/registration/repo/:service_repo_name
Returns list of non expired hosts for `:service_repo_name` (query based on secondary index, for example, DynamoDB GSI).
Format is the same as [query based on service](#get-v1registrationservice).

### POST /v1/registration/:service
Registers a host with a service. Response body does not contain any data.

* service
* *(required, string)* Service for which operation is performed.

Request params:
* ip
* *(required, string)* ip address of the host.
* service_repo_name
* *(optional, string)* service repository name, can be used for quick search.
* port
* *(required, integer)* port on which the host expects connections, Envoy will connect to this port.
* revision
* *(required, string)* SHA of the revision the service is currently running.
* tags
* *(required, object)* JSON in the following [format](#tags-json).

### DELETE /v1/registration/:service/:ip_address
Deletes the host for the given `service` with `ip_address`.
Returns response code 400 if no `service`/`ip_address` entity exists.

* service
* *(required, string)* name of the service metadata is queried for.
* ip
* *(required, string)* ip address of the host.

### POST /v1/loadbalancing/:service/:ip_address
Updates the weight of hosts for load balancing purposes.

* service
* *(required, string)* Service name for which weight is updated.
* ip_address
* *(optional, string)* IP address of the host for which weight is updated.
If not given, *all* hosts for the given service will have their weights updated.

Request params:
* load_balancing_weight
* *(required, integer)* Host weight, an integer between 1 and 100.

#### Tags JSON
```json
{
"az": "...",
"region": "...",
"instance_id": "...",
"canary": false,
"load_balancing_weight": 1
}
```

* az
* *(required, string)* AWS availability zone that the host is running in. You
can provide arbitrary but the same value for all hosts if zone aware stats/routing
is not required. If you use non AWS backend storage currently you have to provide `az`
anyway, you can use some fixed predefined value for all hosts.
* region
* *(required, string)* AWS region that the host is running in.
* instance_id
* *(required, string)* AWS instance_id of the host.
* canary
* *(optional, boolean)* Set to true if host is a canary instance, used by Envoy.
* load_balancing_weight:
* *(optional, integer)* Load balancing weight is used by Envoy for weighted routing.
Values must be an integer between 1 and 100.

## Main Classes
- [app/routes/api.py](https://github.com/lyft/discovery/blob/master/app/routes/api.py)
- defines the HTTP routes for service registration
- [app/resources/api.py](https://github.com/lyft/discovery/blob/master/app/resources/api.py)
- the entry point for service registration GETs and POSTs
- [app/services/host.py](https://github.com/lyft/discovery/blob/master/app/services/host.py)
- contains the business logic (e.g. caching, input validation, etc) for service registration GETS and POSTs
- [app/models/host.py](https://github.com/lyft/discovery/blob/master/app/models/host.py)
- The pynamo model for service registration information for a host

## Unit Testing
*Note* currently it's not working on public repository without tweaking (there is an opened issue for this)
To run all unit tests, run `make test_unit`.