Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/nandoflorestan/kerno

Framework for a Python application service layer, approximating Uncle Bob Martin's Clean Architecture.
https://github.com/nandoflorestan/kerno

application-framework clean-architecture library python python3 tdd

Last synced: 10 days ago
JSON representation

Framework for a Python application service layer, approximating Uncle Bob Martin's Clean Architecture.

Awesome Lists containing this project

README 

        
=====

Kerno

=====



Kerno is trying to be:



- a framework for building applications in Python

- which approximates Robert C. Martin's

  `Clean Architecture `_

- by empowering a service layer (here called **Action layer**)

- and moving both persistence and UI (including web frameworks) to

  the edges of the system, while still providing ways to make

  development and automated testing easier.



The documentation is at http://docs.nando.audio/kerno/latest/



Defense of the architecture

~~~~~~~~~~~~~~~~~~~~~~~~~~~



This is the first such project in the entire Python community, which

seems so addicted to the insufficient MVC or MVT proposed by common

web frameworks. Kerno does not replace these frameworks, but facilitates

creation of the top layers of an architecture that contains,

from center to border:



- **Configuration layer:** Such that all other layers have access to settings.

  Why should configuration settings be available only to the web framework???

- **Model layer:** in which you define entities (e. g. SQLAlchemy models),

  but no I/O (no session, no queries). The entities can be seen

  by all the layers below.

- **Action layer:** in which you define business rules (decisions,

  the core of your app), meaning it can import and use the models above,

  and use other utilities that are dependency-injected.

- **Repository layer:** Performs database access (e. g. queries) and is

  dependency-injected into the Action layer, making unit tests easier to write.

- **Controller layer:** Thin glue layer, the only one that imports and uses

  a web framework or GUI framework.

- **Template layer:** If using a web framework.



In order to better understand why this architecture is good for medium to

large applications, I encourage you to watch a couple of Uncle Bob Martin's

talks on this subject -- easily found on the Internet.



Good software architecture is about decoupling. The main message of the Clean

Architecture seems to be, decouple your business logic from any dependencies,

so your large project can survive these (swapping them with less effort).



Kerno is just a set of whatever tools might have been missing for you to

build the core of your app in that way. You can use only the parts you value.

In fact, it doesn't matter if you use the Kerno library, what is important is

for you to understand:



- that automated testing is necessary today (non-negotiable),

- that function purity makes automated testing possible or easy,

- that decoupling is important for the future of your project,

- that global variables (module-level attributes) make testing difficult

  and error prone, limit concurrency, and increase implementation complexity,

- and that finding the right abstractions can be difficult.



The main inspiration for the *implementation* of Kerno is the Pyramid

web framework because it is so well architected. It managed to avoid

the global variables that plague Flask and Django while providing the

best application composition method of all these frameworks. Sometimes I

think Kerno is doing little more than bringing parts of Pyramid out of the

web framework in a composable way.



We require Python >= 3.6 because we are type annotating our functions so you

can use mypy if you like.



Action (or Service) layer

=========================



Now I am going to tell you why you should decouple 1) your web framework

and 2) your data persistence.



Do not conflate the core of your app -- the business rules -- with usage

of a web framework. You may want to switch web frameworks in the future,

or build an entirely different UI. Therefore, business rules must not

reside in controllers -- move your application's decisions to a higher layer!

Conversely, the layer that contains business rules (the Action layer)

may NOT import anything from the web framework.



http://programmers.stackexchange.com/questions/162399/how-essential-is-it-to-make-a-service-layer



Your controllers should be thin -- just get the data and call an Action.



Also do not conflate business rules with data persistence. If you keep

I/O concerns out of your Service layer, here called Action layer,

then this layer becomes truly unit-testable, which in itself is reason

enough to do this. Integrated test (meaning a test that hits a database,

even SQLite in memory) suites easily take 5+ minutes to run for large apps,

making TDD (test first) impossible. The solution is to separate decisions

in pure functions (which do no I/O) so they can be unit-tested. This way the

unit tests run instantaneously and you are able to do TDD.



MVC or MVT is missing a Service layer between the Model and the Controller.

This layer, here called Action, should contain the business rules in a pure

way, leaving UI related preoccupations to a thin controller, and storage

concerns to Models and the...



Repository layer

================



MVT, as implemented by most Python web frameworks, puts a

data persistence layer in the center of the architecture,

but Uncle Bob's Clean Architecture teaches us this is wrong.



In our solution, the model entities may remain in the center and circulate

(as data holders) through all layers -- this has always been so convenient --,

but the engine that stores and retrieves entities -- here called Repository

(in SQLAlchemy it's the session and its queries) -- must be a

dependency-injected layer in order to make testing easy.



Fowler and Robert C. Martin would have us put the SQLAlchemy models away too,

so in the future you could more easily swap SQLAlchemy with something else,

or even develop multiple storages simultaneously: SQLAlchemy, ZODB etc.,

but this piece of advice I am not following yet.



(Even so, you might be able to pull this off right now, if you remember that

you don't have to use the declarative flavor of the SQLAlchemy ORM --

you can instead declare tables, and then classes, and then mappers that

map tables to the classes.  These classes need no specific base class,

so you are free to use them as your entities.)



Since many large apps are assembled from smaller packages, we have devised

a sort of plugin system that composes the final Repository class from

mixin classes.



SQLAlchemy strategy

-------------------



Functions are easiest to unit-test when they do not perform IO. Any IO you do

is something that needs to be mocked in tests, and mocking is hard.



SQLAlchemy is an excellent example of this. It creates profound testability

challenges for any code that uses it, because its fluent API is very hard

to mock out.



After struggling with the problem for years, we decided that any I/O must

be cleanly decoupled from the Action layer, since this is the most

important layer to be unit-tested. So we follow these rules:



1. The session must be present in the Repository layer, which is

   dependency-injected into the Action layer.  This allows you to write

   fast unit tests for the Action layer -- by injecting a

   FakeRepository object which never touches an RDBMS.

2. The session must NOT be present in the model layer (which defines entities).

   Usage of SQLAlchemy relationships (e.g. ``User.addresses``), though very

   convenient, makes code hard to test because it is doing I/O.

   ``object_session(self)`` also must be avoided to keep the separation.

   For now, I think relationships can continue to exist in models normally,

   but they must be used only in the repository.

3. The session must NOT be imported in the Action layer (which contains

   business rules). Really, only your Repository object can use the session.



Using Kerno

~~~~~~~~~~~



If you wish to adopt the Clean Architecture, then Kerno can help you.

Here is how.



Startup time and request time

=============================



Kerno computes some things at startup and keeps the result in a "global" object

which is an instance of the Kerno class. This instance is initialized with

the app's settings and utilities (strategies) are registered on it.



Then each request uses that to obtain globals and calls an Action.



Component registration

======================



In order to swap components between environments, Kerno could have used the

famous and awesome

`Zope Component Architecture `_,

but we are using `Reg `_ instead.

Reg is very powerful and you don't need to create an interface for

each component you want to register.



However, there's only a certain amount of overlap on the problems solved

by Reg and the ZCA. Reg implements multiple dispatch for functions. The ZCA

aids you with contracts and uses these for multiple dispatch.



Actions

=======



You can express Kerno actions (the service layer) as functions or as classes.

Kerno provides a base class for this purpose.



Web framework integration

=========================



Kerno is trying to provide a good scheme to communicate with web frameworks

in general.



Integration with Pyramid is provided, but totally decoupled and optional.

It includes an Exception class, a view that catches and renders it,

and conventions for returned objects.