Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/adamalton/asynchrormous

A library which adds functionality for asynchronous database operations to Django's ORM
https://github.com/adamalton/asynchrormous

Last synced: about 2 months ago
JSON representation

A library which adds functionality for asynchronous database operations to Django's ORM

Awesome Lists containing this project

README 

        
AsynchrORMous

=============



A library which adds functionality for asynchronous database operations to Django's ORM



## Usage



###### 1. Create your model using the AsyncManager



```

from django.db import models

from asynchrormous import AsyncManager



class MyModel(models.Model):

    objects = AsyncManager() #This gives us async-capable querysets

    field1 = models.CharField(max_length=100)

```



###### 2. Create your queryset as normal.



Note that you do not want to  evaluate the queryset yet, as that would default the point of having the async magic.



```

queryset = MyModel.objects.filter(field1='cake')

```



###### 3. Trigger the asynchronous database operation for whichever evaulation you require.



There are 3 possible database operations which you can trigger:



1. Fetching of results using `.start_fetch()`.  This is for when you plan to iterate over the results or call `len()` on the queryset.

2. Fetching of the count using `start_count()`.  This is for when you plan to call `.count()` on the queryset.

3. Fetching of the existence/non-zero-ness using `start_exists()`.  This is for when you plan to call `.exists()` on the queryset.



The relevant database operation will then be started in the background, allowing your application to continue to do other work while the database operation is in progress.



###### 4. Use your queryset as normal to get the results:



You can now iterate over your queryset, call `.count()`, `.exists()` or `len()` on it as normal.  If the database operation which you triggered earlier has not yet finished then it will now block (as Django normally would) until it has finished.



### Examples



```

queryset = MyModel.objects.filter(field1='cake')[:10].start_fetch()

do_other_things() #This happens in parallel to the database operation

for obj in queryset:

    print object



queryset = MyModel.objects.filter(field1='sausage').start_count()

do_other_things() #This happens in parallel to the database operation

if queryset.count() > 2:

    print "We have many sausages"



queryset = MyModel.objects.filter(field1='miracle').start_exists()

do_other_things() #This happens in parallel to the database operation

if queryset.exists():

    print "We have a miracle!"

```



The most common use case for this functionality is probably going to be the scenario of preparing a queryset in a view and then passing it to the template for rendering the results into one of your fancy web pages.  While the template system is doing its thing parsing the template files, creating the nodes, rendering the header, etc, your database can be happily fetching its results.  Then when the template gets down to `{% for x in results %}` the results will have already been fetched.  BOOM!



### Notes



* For convenience, the `start_()` methods return the queryset.  But unlike `.filter()` and `.all()` they don't create a new clone of the queryset.



## TODO



* Add an `async_get()` method.  If you look in `django.db.models.query.QuerySet.get` you'll see why this would be a good idea.

* Add more examples, including calling `len()` and `bool()` on the queryset, using `.exists()` after calling `.start_fetch()` (which would work), etc.

* Write more tests which check:

  - That the threading actually does what we think it does.  (It works as expected, but I would like more solid proof of this.)

  - More combinations of things like calling `.start_fetch()` followed by `.exists()` (which should use the result of the fetch), and various other fun ways of using it.

* Change the existing tests so that they don't create large numbers of objects.