Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/rochacbruno/nameko-example
Code for example in this post: http://brunorocha.org/python/microservices-with-python-rabbitmq-and-nameko.html
https://github.com/rochacbruno/nameko-example
Last synced: 13 days ago
JSON representation
Code for example in this post: http://brunorocha.org/python/microservices-with-python-rabbitmq-and-nameko.html
- Host: GitHub
- URL: https://github.com/rochacbruno/nameko-example
- Owner: rochacbruno
- Created: 2016-06-29T17:19:14.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2016-09-08T20:29:38.000Z (about 8 years ago)
- Last Synced: 2024-10-25T00:19:17.868Z (20 days ago)
- Language: Python
- Size: 9.77 KB
- Stars: 191
- Watchers: 10
- Forks: 64
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
TL;DR: start coding [here](https://github.com/rochacbruno/nameko-example/blob/master/service/README.md)
> "Micro-services is the new black" - Splitting the project in to independently scalable services is the currently the best option to ensure the evolution of the code. In Python there is a Framework called "Nameko" which makes it very easy and powerful.
# Micro services
> The term "Microservice Architecture" has sprung up over the last few years to describe a particular way of designing software applications as suites of independently deployable services. - M. Fowler
I recommend reading the [Fowler's posts](http://martinfowler.com/articles/microservices.html) to understand the theory behind it.
## Ok I so what does it mean?
In brief a **Micro Service Architecture** exists when your system is divided in small (single context bound) responsibilities blocks, those blocks doesn't know each other, they only have a common point of communication, generally a **message queue**, and does know the communication protocol and interfaces.
## Give me a real-life example
> The code is available on github: [http://github.com/rochacbruno/nameko-example](http://github.com/rochacbruno/nameko-example) take a look at **service** and **api** folders for more info.
Consider you have an REST API, that API has an **endpoint** receiving some data and you need to perform some kind of computation with that data, instead of blocking the caller you can do it asynchronously, return an status "OK - Your request will be processed" to the caller and do it in a background task.
Also you want to send an email notification when the computation is finished without blocking the main computing process, so it is better to delegate the "email sending" to another service.
### Scenario
# Show me the code!
Lets create the system to understand it in practice.
## Environment
We need an environment with:
- A running RabbitMQ
- Python VirtualEnv for services
- Python VirtualEnv for API
### Rabbit
The easiest way to have a RabbitMQ in development environment is running its official docker container, considering you have Docker installed run:
```
docker run -d --hostname my-rabbit --name some-rabbit -p 15672:15672 -p 5672:5672 rabbitmq:3-management
```
Go to the browser and access http://localhost:15672 using credentials **guest:guest** if you can login to RabbitMQ dashboard it means you have it running locally for development.
## The Service environment
Now lets create the Micro Services to consume our tasks. We'll have a service for **computing** and another for **mail**, follow the steps.
In a shell create the root project directory
```
$ mkdir myproject
$ cd myproject
```
Create and activate a virtualenv (you can also use virtualenv-wrapper)
```
$ virtualenv service_env
$ source service_env/bin/activate
```
Install nameko framework and yagmail
```
(service_env)$ pip install nameko
(service_env)$ pip install yagmail
```
## The service code
Now having that virtualenv prepared (consider you can run service in a server and API in another) lets code the nameko RPC Services.
We are going to put both services in a single python module, but you can also split in separate modules and also run them in separate servers if needed.
In a file called ``service.py``
```python
import yagmail
from nameko.rpc import rpc, RpcProxy
class Mail(object):
name = "mail"
@rpc
def send(self, to, subject, contents):
yag = yagmail.SMTP('[email protected]', 'mypassword')
# read the above credentials from a safe place.
# Tip: take a look at Dynaconf setting module
yag.send(to=to.encode('utf-8'),
subject=subject.encode('utf-8'),
contents=[contents.encode('utf-8')])
class Compute(object):
name = "compute"
mail = RpcProxy('mail')
@rpc
def compute(self, operation, value, other, email):
operations = {'sum': lambda x, y: int(x) + int(y),
'mul': lambda x, y: int(x) * int(y),
'div': lambda x, y: int(x) / int(y),
'sub': lambda x, y: int(x) - int(y)}
try:
result = operations[operation](value, other)
except Exception as e:
self.mail.send.async(email, "An error occurred", str(e))
raise
else:
self.mail.send.async(
email,
"Your operation is complete!",
"The result is: %s" % result
)
return result
```
Now with the above services definition we need to run it as a Nameko RPC service.
> NOTE: We are going to run it in a console and leave it running, but in production it is recommended to put the service to run using **supervisord** or an alternative.
## Run the service and let it running in a shell
```
(service_env)$ nameko run service --broker amqp://guest:guest@localhost
starting services: mail, compute
Connected to amqp://guest:**@127.0.0.1:5672//
Connected to amqp://guest:**@127.0.0.1:5672//
```
## Testing it
Go to another shell (with the same virtenv) and test it using ``nameko shell``
```
(service_env)$ nameko shell --broker amqp://guest:guest@localhost
Nameko Python 2.7.9 (default, Apr 2 2015, 15:33:21)
[GCC 4.9.2] shell on linux2
Broker: amqp://guest:guest@localhost
>>>
```
You are now in the RPC client testing shell exposing the **n.rpc** object, play with it
```
>>> n.rpc.mail.send("[email protected]", "testing", "Just testing")
```
The above should sent an email and we can also call **compute** service to test it, note that it also spawns an async **mail** sending with result.
```
>>> n.rpc.compute.compute('sum', 30, 10, "[email protected]")
40
>>> n.rpc.compute.compute('sub', 30, 10, "[email protected]")
20
>>> n.rpc.compute.compute('mul', 30, 10, "[email protected]")
300
>>> n.rpc.compute.compute('div', 30, 10, "[email protected]")
3
```
# Calling the micro-service through the API
In a different shell (or even a different server) prepare the API environment
Create and activate a virtualenv (you can also use virtualenv-wrapper)
```
$ virtualenv api_env
$ source api_env/bin/activate
```
Install Nameko, Flask and Flasgger
```
(api_env)$ pip install nameko
(api_env)$ pip install flask
(api_env)$ pip install flasgger
```
> NOTE: In api you dont need the **yagmail** because it is **service** responsability
Lets say you have the following code in a file ``api.py``
```python
from flask import Flask, request
from flasgger import Swagger
from nameko.standalone.rpc import ClusterRpcProxy
app = Flask(__name__)
Swagger(app)
CONFIG = {'AMQP_URI': "amqp://guest:guest@localhost"}
@app.route('/compute', methods=['POST'])
def compute():
"""
Micro Service Based Compute and Mail API
This API is made with Flask, Flasgger and Nameko
---
parameters:
- name: body
in: body
required: true
schema:
id: data
properties:
operation:
type: string
enum:
- sum
- mul
- sub
- div
email:
type: string
value:
type: integer
other:
type: integer
responses:
200:
description: Please wait the calculation, you'll receive an email with results
"""
operation = request.json.get('operation')
value = request.json.get('value')
other = request.json.get('other')
email = request.json.get('email')
msg = "Please wait the calculation, you'll receive an email with results"
subject = "API Notification"
with ClusterRpcProxy(CONFIG) as rpc:
# asynchronously spawning and email notification
rpc.mail.send.async(email, subject, msg)
# asynchronously spawning the compute task
result = rpc.compute.compute.async(operation, value, other, email)
return msg, 200
app.run(debug=True)
```
Put the above API to run in a different shell or server
```
(api_env) $ python api.py
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
```
and then access the url http://localhost:5000/apidocs/index.html you will see the Flasgger UI and you can interact with the api and start producing tasks on queue to the service to consume.
![[image]](http://brunorocha.org/static/media/microservices/Flasgger_API_documentation.png "enter image title here")
> NOTE: You can see the shell where service is running for logging, prints and error messages. You can also access the RabbitMQ dashboard to see if there is some message in process there.
There is a lot of more advanced things you can do with Nameko framework you can find more information on https://nameko.readthedocs.org/en/stable/
Let's Micro Serve!