Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/simonw/asyncinject
Run async workflows using pytest-fixtures-style dependency injection
https://github.com/simonw/asyncinject
asyncio
Last synced: 11 days ago
JSON representation
Run async workflows using pytest-fixtures-style dependency injection
- Host: GitHub
- URL: https://github.com/simonw/asyncinject
- Owner: simonw
- License: apache-2.0
- Created: 2021-11-17T01:35:15.000Z (almost 3 years ago)
- Default Branch: main
- Last Pushed: 2023-04-14T01:28:28.000Z (over 1 year ago)
- Last Synced: 2024-10-18T07:53:10.992Z (26 days ago)
- Topics: asyncio
- Language: Python
- Homepage:
- Size: 37.1 KB
- Stars: 34
- Watchers: 3
- Forks: 2
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# asyncinject
[](https://pypi.org/project/asyncinject/)
[](https://github.com/simonw/asyncinject/releases)
[](https://github.com/simonw/asyncinject/blob/main/LICENSE)
Run async workflows using pytest-fixtures-style dependency injection
## Installation
Install this library using `pip`:
$ pip install asyncinject
## Usage
This library is inspired by [pytest fixtures](https://docs.pytest.org/en/6.2.x/fixture.html).
The idea is to simplify executing parallel `asyncio` operations by allowing them to be defined using a collection of functions, where the function arguments represent dependent functions that need to be executed first.
The library can then create and execute a plan for executing the required functions in parallel in the most efficient sequence possible.
Here's an example, using the [httpx](https://www.python-httpx.org/) HTTP library.
```python
from asyncinject import Registry
import httpx
async def get(url):
async with httpx.AsyncClient() as client:
return (await client.get(url)).text
async def example():
return await get("http://www.example.com/")
async def simonwillison():
return await get("https://simonwillison.net/search/?tag=empty")
async def both(example, simonwillison):
return example + "\n\n" + simonwillison
registry = Registry(example, simonwillison, both)
combined = await registry.resolve(both)
print(combined)
```
If you run this in `ipython` or `python -m asyncio` (to enable top-level await in the console) you will see output that combines HTML from both of those pages.
The HTTP requests to `www.example.com` and `simonwillison.net` will be performed in parallel.
The library notices that `both()` takes two arguments which are the names of other registered `async def` functions, and will construct an execution plan that executes those two functions in parallel, then passes their results to the `both()` method.
### Registry.from_dict()
Passing a list of functions to the `Registry` constructor will register each function under their introspected function name, using `fn.__name__`.
You can set explicit names instead using a dictionary:
```python
registry = Registry.from_dict({
"example": example,
"simonwillison": simonwillison,
"both": both
})
```
Those string names will be used to match parameters, so each function will need to accept parameters named after the keys used in that dictionary.
### Registering additional functions
Functions that are registered can be regular functions or `async def` functions.
In addition to registering functions by passing them to the constructor, you can also add them to a registry using the `.register()` method:
```python
async def another():
return "another"
registry.register(another)
```
To register them with a name other than the name of the function, pass the `name=` argument:
```python
async def another():
return "another 2"
registry.register(another, name="another_2")
```
### Resolving an unregistered function
You don't need to register the final function that you pass to `.resolve()` - if you pass an unregistered function, the library will introspect the function's parameters and resolve them directly.
This works with both regular and async functions:
```python
async def one():
return 1
async def two():
return 2
registry = Registry(one, two)
# async def works here too:
def three(one, two):
return one + two
print(await registry.resolve(three))
# Prints 3
```
### Parameters are passed through
Your dependent functions can require keyword arguments which have been passed to the `.resolve()` call:
```python
async def get_param_1(param1):
return await get(param1)
async def get_param_2(param2):
return await get(param2)
async def both(get_param_1, get_param_2):
return get_param_1 + "\n\n" + get_param_2
combined = await Registry(get_param_1, get_param_2, both).resolve(
both,
param1 = "http://www.example.com/",
param2 = "https://simonwillison.net/search/?tag=empty"
)
print(combined)
```
### Parameters with default values are ignored
You can opt a parameter out of the dependency injection mechanism by assigning it a default value:
```python
async def go(calc1, x=5):
return calc1 + x
async def calc1():
return 5
print(await Registry(calc1, go).resolve(go))
# Prints 10
```
### Tracking with a timer
You can pass a `timer=` callable to the `Registry` constructor to gather timing information about executed tasks.. Your function should take three positional arguments:
- `name` - the name of the function that is being timed
- `start` - the time that it started executing, using `time.perf_counter()` ([perf_counter() docs](https://docs.python.org/3/library/time.html#time.perf_counter))
- `end` - the time that it finished executing
You can use `print` here too:
```python
combined = await Registry(
get_param_1, get_param_2, both, timer=print
).resolve(
both,
param1 = "http://www.example.com/",
param2 = "https://simonwillison.net/search/?tag=empty"
)
```
This will output:
```
get_param_1 436633.584580685 436633.797921747
get_param_2 436633.641832699 436634.196364347
both 436634.196570217 436634.196575639
```
### Turning off parallel execution
By default, functions that can run in parallel according to the execution plan will run in parallel using `asyncio.gather()`.
You can disable this parallel exection by passing `parallel=False` to the `Registry` constructor, or by setting `registry.parallel = False` after the registry object has been created.
This is mainly useful for benchmarking the difference between parallel and serial execution for your project.
## Development
To contribute to this library, first checkout the code. Then create a new virtual environment:
cd asyncinject
python -m venv venv
source venv/bin/activate
Now install the dependencies and test dependencies:
pip install -e '.[test]'
To run the tests:
pytest