Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

https://github.com/aiokitchen/aiomisc

aiomisc - miscellaneous utils for asyncio
https://github.com/aiokitchen/aiomisc

aiomisc async-await async-python asynchronous asyncio colorlogger miscellaneous server-client service uvloop

Last synced: 2 months ago
JSON representation

aiomisc - miscellaneous utils for asyncio

Lists

README 

        
aiomisc - miscellaneous utils for asyncio

=========================================



.. image:: https://coveralls.io/repos/github/aiokitchen/aiomisc/badge.svg?branch=master

   :target: https://coveralls.io/github/aiokitchen/aiomisc

   :alt: Coveralls



.. image:: https://github.com/aiokitchen/aiomisc/workflows/tox/badge.svg

   :target: https://github.com/aiokitchen/aiomisc/actions?query=workflow%3Atox

   :alt: Actions



.. image:: https://img.shields.io/pypi/v/aiomisc.svg

   :target: https://pypi.python.org/pypi/aiomisc/

   :alt: Latest Version



.. image:: https://img.shields.io/pypi/wheel/aiomisc.svg

   :target: https://pypi.python.org/pypi/aiomisc/



.. image:: https://img.shields.io/pypi/pyversions/aiomisc.svg

   :target: https://pypi.python.org/pypi/aiomisc/



.. image:: https://img.shields.io/pypi/l/aiomisc.svg

   :target: https://pypi.python.org/pypi/aiomisc/



Miscellaneous utils for asyncio.



As a programmer, you are no stranger to the challenges that come with building

and maintaining software applications. One area that can be particularly

difficult is making architecture of the software that using asynchronous I/O.



This is where aiomisc comes in. aiomisc is a Python library that provides a

collection of utility functions and classes for working with asynchronous I/O

in a more intuitive and efficient way. It is built on top of the ``asyncio``

library and is designed to make it easier for developers to write

asynchronous code that is both reliable and scalable.



With aiomisc, you can take advantage of powerful features like

``worker pools``, ``connection pools``, ``circuit breaker pattern``,

and retry mechanisms such as ``asyncbackoff`` and ``asyncretry`` to make your

asyncio code more robust and easier to maintain. In this documentation,

we'll take a closer look at what ``aiomisc`` has to offer and how it can

help you streamline your asyncio service development.



Installation

------------



Installation is possible in standard ways, such as PyPI or installation from

a git repository directly.



Installing from PyPI_:



.. code-block:: bash



    pip3 install aiomisc



Installing from github.com:



.. code-block:: bash



    # Using git tool

    pip3 install git+https://github.com/aiokitchen/aiomisc.git



    # Alternative way using http

    pip3 install \

        https://github.com/aiokitchen/aiomisc/archive/refs/heads/master.zip



The package contains several extras and you can install additional dependencies

if you specify them in this way.



With uvloop_:



.. code-block:: bash



    pip3 install "aiomisc[uvloop]"



With aiohttp_:



.. code-block:: bash



    pip3 install "aiomisc[aiohttp]"



Complete table of extras bellow:



+-----------------------------------+------------------------------------------------+

| example                           |  description                                   |

+===================================+================================================+

| ``pip install aiomisc[aiohttp]``  | For running aiohttp_ applications.             |

+-----------------------------------+------------------------------------------------+

| ``pip install aiomisc[asgi]``     | For running ASGI_ applications                 |

+-----------------------------------+------------------------------------------------+

| ``pip install aiomisc[carbon]``   | Sending metrics to carbon_ (part of graphite_) |

+-----------------------------------+------------------------------------------------+

| ``pip install aiomisc[cron]``     | use croniter_ for scheduling tasks             |

+-----------------------------------+------------------------------------------------+

| ``pip install aiomisc[raven]``    | Sending exceptions to sentry_ using raven_     |

+-----------------------------------+------------------------------------------------+

| ``pip install aiomisc[rich]``     | You might using rich_ for logging              |

+-----------------------------------+------------------------------------------------+

| ``pip install aiomisc[uvicorn]``  | For running ASGI_ application using uvicorn_   |

+-----------------------------------+------------------------------------------------+

| ``pip install aiomisc[uvloop]``   | use uvloop_ as a default event loop            |

+-----------------------------------+------------------------------------------------+



.. _ASGI: https://asgi.readthedocs.io/en/latest/

.. _PyPI: https://pypi.org/

.. _aiohttp: https://pypi.org/project/aiohttp

.. _carbon: https://pypi.org/project/carbon

.. _croniter: https://pypi.org/project/croniter

.. _graphite: http://graphiteapp.org

.. _raven: https://pypi.org/project/raven

.. _rich: https://pypi.org/project/rich

.. _sentry: https://sentry.io/

.. _uvloop: https://pypi.org/project/uvloop

.. _uvicorn: https://pypi.org/project/uvicorn



You can combine extras values by separating them with commas, for example:



.. code-block:: bash



    pip3 install "aiomisc[aiohttp,cron,rich,uvloop]"



Quick Start

-----------



This section will cover how this library creates and uses the event loop and

creates services. Of course, you can't write about everything here, but you

can read about a lot in the Tutorial_ section, and you can

always refer to the Modules_ and `API reference`_ sections for help.



Event-loop and entrypoint

+++++++++++++++++++++++++



Let's look at this simple example first:



.. code-block:: python



    import asyncio

    import logging



    import aiomisc



    log = logging.getLogger(__name__)



    async def main():

        log.info('Starting')

        await asyncio.sleep(3)

        log.info('Exiting')



    if __name__ == '__main__':

        with aiomisc.entrypoint(log_level="info", log_format="color") as loop:

            loop.run_until_complete(main())



This code declares an asynchronous ``main()`` function that exits after

3 seconds. It would seem nothing interesting, but the whole point is in

the ``entrypoint``.



What does the ``entrypoint`` do, it would seem not so much, it creates an

event-loop and transfers control to the user. However, under the hood, the

logger is configured in a separate thread, a pool of threads is created,

services are started, but more on that later and there are no services

in this example.



Alternatively, you can choose not to use an entrypoint, just create an

event-loop and set this as a default event loop for current thread:



.. code-block:: python

    :name: test_index_get_loop



    import asyncio

    import aiomisc



    # * Installs uvloop event loop is it's has been installed.

    # * Creates and set `aiomisc.thread_pool.ThreadPoolExecutor`

    #   as a default executor

    # * Sets just created event-loop as a current event-loop for this thread.

    aiomisc.new_event_loop()



    async def main():

        await asyncio.sleep(1)



    if __name__ == '__main__':

        loop = asyncio.get_event_loop()

        loop.run_until_complete(main())



The example above is useful if your code is already using an implicitly created

event loop, you will have to modify less code, just add

``aiomisc.new_event_loop()`` and all calls to ``asyncio.get_event_loop()``

will return the created instance.



However, you can do with one call. Following example closes implicitly created

asyncio event loop and install a new one:



.. code-block:: python

    :name: test_index_new_loop



    import asyncio

    import aiomisc



    async def main():

        await asyncio.sleep(3)



    if __name__ == '__main__':

        loop = aiomisc.new_event_loop()

        loop.run_until_complete(main())



Services

++++++++



The main thing that an ``entrypoint`` does is start and gracefully

stop services.



The service concept within this library means a class derived from

the ``aiosmic.Service`` class and implementing the

``async def start(self) -> None:`` method and optionally the

``async def stop(self, exc: Optional[ Exception]) -> None`` method.



The concept of stopping a service is not necessarily is pressing ``Ctrl+C``

keys by user, it's actually just exiting the ``entrypoint`` context manager.



The example below shows what your service might look like:



.. code-block:: python



    from aiomisc import entrypoint, Service



    class MyService(Service):

        async def start(self):

            do_something_when_start()



        async def stop(self, exc):

            do_graceful_shutdown()



    with entrypoint(MyService()) as loop:

        loop.run_forever()



The entry point can start as many instances of the service as it likes,

and all of them will start concurrently.



There is also a way if the ``start`` method is a payload for a service,

and then there is no need to implement the stop method, since the running

task with the ``start`` function will be canceled at the stop stage.

But in this case, you will have to notify the ``entrypoint`` that the

initialization of the service instance is complete and it can continue.



Like this:



.. code-block:: python



    import asyncio

    from threading import Event

    from aiomisc import entrypoint, Service



    event = Event()



    class MyService(Service):

        async def start(self):

            # Send signal to entrypoint for continue running

            self.start_event.set()

            await asyncio.sleep(3600)



    with entrypoint(MyService()) as loop:

        assert event.is_set()



.. note::



    The ``entrypoint`` passes control to the body of the context manager only

    after all service instances have started. As mentioned above, a start is

    considered to be the completion of the ``start`` method or the setting of

    an start event with ``self.start_event.set()``.



The whole power of this library is in the set of already implemented or

abstract services.

Such as: ``AIOHTTPService``, ``ASGIService``, ``TCPServer``,

``UDPServer``, ``TCPClient``, ``PeriodicService``, ``CronService`` and so on.



Unfortunately in this section it is not possible to pay more attention to this,

please pay attention to the Tutorial_ section section, there are more

examples and explanations, and of cource you always can find out an answer on

the `/api/index` or in the source code. The authors have tried to make

the source code as clear and simple as possible, so feel free to explore it.



Versioning

----------



This software follows `Semantic Versioning`_



Summary: it's given a version number MAJOR.MINOR.PATCH, increment the:



* MAJOR version when you make incompatible API changes

* MINOR version when you add functionality in a backwards compatible manner

* PATCH version when you make backwards compatible bug fixes

* Additional labels for pre-release and build metadata are available as

  extensions to the MAJOR.MINOR.PATCH format.



In this case, the package version is assigned automatically with poem-plugins_,

it using on the tag in the repository as a major and minor and the counter,

which takes the number of commits between tag to the head of branch.



.. _poem-plugins: https://pypi.org/project/poem-plugins



Summary: it's given a version number MAJOR.MINOR.PATCH, increment the:



* MAJOR version when you make incompatible API changes

* MINOR version when you add functionality in a backwards compatible manner

* PATCH version when you make backwards compatible bug fixes

* Additional labels for pre-release and build metadata are available as

  extensions to the MAJOR.MINOR.PATCH format.



In this case, the package version is assigned automatically with poem-plugins_,

it using on the tag in the repository as a major and minor and the counter,

which takes the number of commits between tag to the head of branch.



.. _poem-plugins: https://pypi.org/project/poem-plugins



How to develop?

---------------



This project, like most open source projects, is developed by enthusiasts,

you can join the development, submit issues, or send your merge requests.



In order to start developing in this repository, you need to do the

following things.



Should be installed:



* Python 3.7+ as ``python3``

* Installed Poetry_ as ``poetry``



.. _Poetry: https://python-poetry.org/docs/



For setting up developer environment just execute:



    .. code-block::



        # installing all dependencies

        poetry install



        # setting up pre-commit hooks

        poetry run pre-commit install



        # adding poem-plugins to the poetry

        poetry self add poem-plugins



.. _Semantic Versioning: http://semver.org/



.. _API reference: https://aiomisc.readthedocs.io/en/latest/api/index.html

.. _Modules: https://aiomisc.readthedocs.io/en/latest/modules.html

.. _Tutorial: https://aiomisc.readthedocs.io/en/latest/tutorial.html