https://github.com/roadsideseb/mti-lightbulb
Testing how different aproaches to multi table inheritance and alternatives through the Django ORM.
https://github.com/roadsideseb/mti-lightbulb
Last synced: 9 months ago
JSON representation
Testing how different aproaches to multi table inheritance and alternatives through the Django ORM.
- Host: GitHub
- URL: https://github.com/roadsideseb/mti-lightbulb
- Owner: roadsideseb
- License: mit
- Created: 2013-09-23T03:11:13.000Z (almost 13 years ago)
- Default Branch: master
- Last Pushed: 2013-12-16T12:50:03.000Z (over 12 years ago)
- Last Synced: 2024-04-16T03:32:53.341Z (about 2 years ago)
- Language: JavaScript
- Size: 582 KB
- Stars: 2
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.rst
- License: LICENSE
Awesome Lists containing this project
README
=============
MTI Lightbulb
=============
.. image:: https://d2weczhvl823v0.cloudfront.net/elbaschid/mti-lightbulb/trend.png
**Disclaimer:** this is a small research project and is currently under
development so there might be changes at any point in time. It is also not
complete and there are things that will break. Use with care :)
Whenever you hear experienced Django developers talk about using Django's
models and define content structures, they tell you that *multi-table
inheritance* (MTI) is bad. Looking at the SQL statements that are generated by
the ORM, you'll see some nasty ``LEFT INNER JOIN`` s in there. The question that
bugged my, however, was **how** bad is it actually, when does it become a real
performance problem and what are the alternatives.
This project is my opinionated attempt to shed some light and provide a some
alternative suggestions for MTI. This is most likely not a generic solution and
it is not trying to be. The structure I used is driven by the project that
triggered this investigative process: `django-fancypages`_.
.. image:: https://raw.github.com/elbaschid/mti-lightbulb/master/docs/_static/images/plot_mti_vs_generic_m2m.png
I'll provide more details on my approach, the results and how to run the tests
as I get some more time to work on this...be patient and check back soon...
.. _`django-fancypages`: https://github.com/tangentlabs/django-fancypages
Setting it all up
-----------------
To try this out, follow these steps to setup you local copy of the projects.
Clone the repo locally::
git clone https://github.com/elbaschid/mti-lightbulb.git
Now install all the dependencies for the project by running::
pip install -r requirments_dev.txt
This will probably take a while since it will also install some dependencies
such as ``numpy`` and ``matplotlib`` for investigating the data.
After the installation you are almost ready to go. I've tried all this in a
vagrant box and setting one up yourself is the easiest way to get going. If you
don't have vagrant or want to use your own setup, change the configuration in
``lightbulb/settings.py`` manually.
To get a working vagrant box up and running make sure that you have vagrant
installed. If you don't follow the installation guide `on the Vagrant
website`_. You also have to make sure that you have the ``precise64`` base box
available and the vagrant plugin for salt provisioning. Get the base box::
vagrant init precise64 http://files.vagrantup.com/precise64.box
and then add the plugin for salt to your vagrant installation::
vagrant plugin install vagrant-salt
Now you can setup a new box, this will take some time since it will install a
few requirements and set up a PostgreSQL database::
vagrant up
You are ready to play with it!
Running the benchmark
---------------------
There are currently two apps available that can be used for testing.
1. ``multi_table``: which tests multi-table inheritance using the
`django-model-utils`_ package
2. ``generic_m2m``: which uses the `django-generic-m2m`_ app to handle a
Many2Many relationship for generic foreign keys.
Running a benchmark on either one of those test apps is implemented in the
``benchy`` app as a management command. Using the basic set up you would run a
test on multi-table inheritance using::
./manage.py run_benchmark multi_table
This will run DB queries against database models starting with 1 model going up
to 500 in steps of 5 (default setting). You can change these by providing
``--num-block 200`` or ``--step-size 1``. By default, a timestamped CSV file is
created and the results of each iteration is printed to the console as well (so
you see some progress).
**Spoiler/Warning:** I've run this on a Digital Ocean box with 1GB of RAM and
the MTI test crashes the DB server when using 100 blocks because it uses up all
the memory. So be careful with that :)
Feedback & Contributions
------------------------
I appreciate any feedback on the way I am approaching this, how it's
implemented. If you have any input on this feel free to file an issue on github
or send me an email.
.. _`on the Vagrant website`: http://docs.vagrantup.com/v2/installation/index.html
.. _`django-model-utils`: https://github.com/carljm/django-model-utils
.. _`django-generic-m2m`: https://github.com/coleifer/django-generic-m2m