{"id":15433148,"url":"https://github.com/simonw/mendoza-trees-workshop","last_synced_at":"2025-04-19T17:53:06.973Z","repository":{"id":66508121,"uuid":"132637695","full_name":"simonw/mendoza-trees-workshop","owner":"simonw","description":"A live-coding workshop illustrating the powerful combination of Jupyter notebooks and Django","archived":false,"fork":false,"pushed_at":"2018-05-08T19:03:48.000Z","size":1132,"stargazers_count":7,"open_issues_count":0,"forks_count":1,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-10-18T07:54:14.496Z","etag":null,"topics":["django","jupyter","tutorial"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/simonw.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-05-08T16:42:56.000Z","updated_at":"2020-10-21T01:19:31.000Z","dependencies_parsed_at":null,"dependency_job_id":"f5827290-6cb6-49d9-ac93-5e4049da5560","html_url":"https://github.com/simonw/mendoza-trees-workshop","commit_stats":{"total_commits":12,"total_committers":1,"mean_commits":12.0,"dds":0.0,"last_synced_commit":"2e2e1cdc4deb81b56cb4077f8bbd3e582938d2e4"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonw%2Fmendoza-trees-workshop","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonw%2Fmendoza-trees-workshop/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonw%2Fmendoza-trees-workshop/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/simonw%2Fmendoza-trees-workshop/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/simonw","download_url":"https://codeload.github.com/simonw/mendoza-trees-workshop/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249753152,"owners_count":21320669,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["django","jupyter","tutorial"],"created_at":"2024-10-01T18:32:07.041Z","updated_at":"2025-04-19T17:53:06.964Z","avatar_url":"https://github.com/simonw.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Mendoza Trees Django Tutorial\n\n**Presented 8th May 2018**. This was primarily a live-coding workshop to illustrate the powerful combination of Jupyter notebooks and Django.\n\n## Initial setup\n\nI started by creating and activating a brand new Python 3 virtual environment:\n\n    $ mkdir mendoza-trees \u0026\u0026 cd mendoza-trees\n    $ python3 -mvenv venv\n    $ source venv/bin/activate\n\nThen I installed [Django](https://www.djangoproject.com/), [django_extensions](https://django-extensions.readthedocs.io/en/latest/), [Jupyter](http://jupyter.org/) and [chardet](https://pypi.org/project/chardet/):\n\n    $ pip install Django django_extensions jupyter chardet\n\nI downloaded a [CSV file full of trees](https://ckan.ciudaddemendoza.gov.ar/dataset/arbolado-de-la-ciudad-de-mendoza) from the Mendoza Open Data site:\n\n    wget http://ckan.ciudaddemendoza.gov.ar/dataset/3aaec520-6c8e-417e-85b6-5c7fc18b353e/resource/1b8b1627-9f6a-4ff6-9422-0ef79e5821bc/download/arboladolineal.csv\n\n## Creating the Django project\n\nI created the Django project like this:\n\n    $ django-admin startproject mendoza_trees\n    $ cd mendoza_trees\n    $ ./manage.py startapp trees\n\nThen I added the following to `trees/models.py`:\n\n    from django.db import models\n\n    class Species(models.Model):\n        name = models.CharField(max_length=100)\n\n        def __str__(self):\n            return self.name\n\n    class Tree(models.Model):\n        species = models.ForeignKey(Species, on_delete=models.CASCADE)\n        latitude = models.FloatField()\n        longitude = models.FloatField()\n\nI added both `django_extensions` and `trees` to `INSTALLED_APPS` in the `mendoza_trees/settings.py` file:\n\n    INSTALLED_APPS = [\n        'django.contrib.admin',\n        'django.contrib.auth',\n        'django.contrib.contenttypes',\n        'django.contrib.sessions',\n        'django.contrib.messages',\n        'django.contrib.staticfiles',\n        'django_extensions',\n        'trees',\n    ]\n\nFinally, I created and ran migrations to create the tables:\n\n    $ ./manage.py makemigrations\n    $ ./manage.py migrate\n\n## Parsing CSV using a Jupyter notebook\n\nPython has numerous more efficient ways to parse CSV files, but I used an interactive Jupyter session to illustrate how iterative programming in Jupyter works.\n\nFirst, I started Jupyter using a new command added by `django_extensions`:\n\n    $ ./manage.py shell_plus --notebook\n\nThis starts a Juyter notebook instance that is already configured to talk to your current Django project.\n\nI chose `New Notebook -\u003e Django Shell-Plus`.\n\nSee my [Import-Trees-from-CSV](https://github.com/simonw/mendoza-trees-workshop/blob/master/Import-Trees-from-CSV.ipynb) notebook for the next steps.\n\n## Showing the trees in the Django Admin\n\nThe Django Admin isn't just for building admin interfaces: I use it on almost all of my projects as a quick visual debugging tool.\n\nI added the following to `trees/admin.py`:\n\n    from django.contrib import admin\n    from .models import Tree, Species\n\n    admin.site.register(Tree)\n    admin.site.register(Species)\n\nNext, I created a superuser:\n\n    $ ./manage.py createsuperuser\n    (set username, email and password)\n\nThen I ran the development server:\n\n    $ ./manage.py runserver\n\nAnd navigated to http://localhost:8000/admin/ and signed in.\n\nThe Trees listing in admin wasn't that useful, so I changed `admin.py` to look like this:\n\n    admin.site.register(Tree,\n        list_display=('species', 'latitude', 'longitude')\n    )\n\nI carried out the above steps while the import script was still running in Jupyter - that way I could hit \"refresh\" in the admin list view and watch the number of trees increase as the import progressed.\n\n## Playing with the Django ORM in Jupyter\n\nHaving loaded all of the trees, I used Jupyter to demonstrate some features of the Django ORM. The full notebook is [Trees-Django-ORM](https://github.com/simonw/mendoza-trees-workshop/blob/master/Trees-Django-ORM.ipynb). The highlights were:\n\n    from trees.models import Species, Tree\n    from django.db.models import Count\n\n    for s in Species.objects.annotate(\n        num_trees = Count('tree')\n    ).order_by('-num_trees'):\n        print(s.name, s.num_trees)\n\n    Morera 18809\n    Fresno europeo 8037\n    Pltano 4319\n    Paraiso 4064\n    N/D 2831\n    Fresno americano 2022\n    Acacia SP 1038\n    Acer 951\n    Paraiso sombrilla 809\n    Olmo comun 776\n    Caducifolio 650\n    Prunas 558\n    Jacarand 523\n    Perenne 470\n    Aguaribay 423\n    Olmo bola 323\n    Ailanthus 299\n    Conifera 252\n    Ligustro 202\n    Tilo 196\n    lamo blanco 167\n    Tipa 127\n    Braquiquito 109\n    lamo criollo 108\n    Acacia visco 108\n    Liquidambar 99\n    Palo borracho 64\n    Catalpa 45\n    Eucalyptus 26\n    Algarrobo 7\n    Arabia 4\n    rbol del cielo 2\n    Maiten 1\n\nI then demonstrated using `django.db.connection` to see exactly what SQL had been executed:\n\n    from django.db import connection\n    connection.queries[-1]\n\n    {'sql': 'SELECT \"trees_species\".\"id\", \"trees_species\".\"name\", COUNT(\"trees_tree\".\"id\") AS \"num_trees\" FROM \"trees_species\" LEFT OUTER JOIN \"trees_tree\" ON (\"trees_species\".\"id\" = \"trees_tree\".\"species_id\") GROUP BY \"trees_species\".\"id\", \"trees_species\".\"name\" ORDER BY \"num_trees\" DESC',\n    'time': '0.020'}\n\nFinally, I demonstrated how the following uses inefficient SQL, executing a SELECT against the species table for every one of the 20 trees that are displayed:\n\n    for tree in Tree.objects.all()[:20]:\n        print(tree.latitude, tree.longitude, tree.species.name)\n\nWhereas the same thing using `.select_related()` only ran one query:\n\n    for tree in Tree.objects.select_related('species')[:20]:\n        print(tree.latitude, tree.longitude, tree.species.name)\n\n## Displaying trees on a map\n\nI built a simple Django view and template that could display some of the trees on a map, using [Leaflet](https://github.com/Leaflet/Leaflet.markercluster) and [Leaflet.markercluster](https://github.com/Leaflet/Leaflet.markercluster). `trees/views.py` looked like this:\n\n    from django.shortcuts import render\n    from .models import Tree\n    import json\n    from django.utils.html import mark_safe\n\n    def index(request):\n        trees = list(Tree.objects.select_related('species').values(\n            'latitude', 'longitude', 'species__name'\n        )[:100])\n        return render(request, 'index.html', {\n            'trees': mark_safe(json.dumps(trees)),\n        })\n\nAnd `trees/templates/index.html` looked like this:\n\n    \u003c!DOCTYPE html\u003e\n    \u003chtml lang=\"en\" dir=\"ltr\"\u003e\n    \u003chead\u003e\n    \u003cmeta charset=\"utf-8\"\u003e\n    \u003ctitle\u003eMendoza Trees\u003c/title\u003e\n    \u003clink rel=\"stylesheet\" href=\"https://unpkg.com/leaflet@1.3.1/dist/leaflet.css\" integrity=\"sha512-Rksm5RenBEKSKFjgI3a41vrjkw4EVPlJ3+OiI65vTjIdo9brlAacEuKOiQ5OFh7cOI1bkDwLqdLw3Zg0cRJAAQ==\" crossorigin=\"anonymous\"\u003e\n    \u003clink rel=\"stylesheet\" href=\"https://unpkg.com/leaflet.markercluster@1.3.0/dist/MarkerCluster.css\" integrity=\"sha384-lPzjPsFQL6te2x+VxmV6q1DpRxpRk0tmnl2cpwAO5y04ESyc752tnEWPKDfl1olr\" crossorigin=\"anonymous\"\u003e\n    \u003clink rel=\"stylesheet\" href=\"https://unpkg.com/leaflet.markercluster@1.3.0/dist/MarkerCluster.Default.css\" integrity=\"sha384-5kMSQJ6S4Qj5i09mtMNrWpSi8iXw230pKU76xTmrpezGnNJQzj0NzXjQLLg+jE7k\" crossorigin=\"anonymous\"\u003e\n    \u003cscript src=\"https://unpkg.com/leaflet@1.3.1/dist/leaflet.js\" integrity=\"sha512-/Nsx9X4HebavoBvEBuyp3I7od5tA0UzAxs+j83KgC8PU0kgB4XiK4Lfe4y4cgBtaRJQEIFCW+oC506aPT2L1zw==\" crossorigin=\"anonymous\"\u003e\u003c/script\u003e\n    \u003cscript src=\"https://unpkg.com/leaflet.markercluster@1.3.0/dist/leaflet.markercluster-src.js\" integrity=\"sha384-NAOEbWFcjnXc7U9GkULPhupHZNAbqru9dS3c+4ANYAwtFoVAWuVuMVDH0DIy4ESp\" crossorigin=\"anonymous\"\u003e\u003c/script\u003e\n    \u003c/head\u003e\n    \u003cbody\u003e\n    \u003cdiv id=\"map\" style=\"width: 95%; height: 90vh\"\u003e\u003c/div\u003e\n    \u003cscript\u003e\n    var trees = {{ trees }};\n    var tiles = L.tileLayer('https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', {\n        maxZoom: 19,\n        detectRetina: true,\n        attribution: '\u0026copy; \u003ca href=\"https://www.openstreetmap.org/copyright\"\u003eOpenStreetMap\u003c/a\u003e contributors, Points \u0026copy 2012 LINZ'\n    });\n    var latlng = L.latLng(-68.816667, -32.883333);\n    var map = L.map('map', {\n        center: latlng,\n        zoom: 13,\n        layers: [tiles]\n    });\n    var currentLayer = null;\n    currentLayer = L.markerClusterGroup({\n        chunkedLoading: true,\n        maxClusterRadius: 50\n    });\n    var markerList = [];\n    trees.forEach(tree =\u003e {\n        var marker = L.marker(L.latLng(tree.latitude, tree.longitude), { title: tree.species });\n        marker.bindPopup(tree.species);\n        markerList.push(marker);\n    });\n    currentLayer.addLayers(markerList);\n    map.addLayer(currentLayer);\n    map.fitBounds(currentLayer.getBounds());\n    \u003c/script\u003e\n    \u003c/body\u003e\n    \u003c/html\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimonw%2Fmendoza-trees-workshop","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsimonw%2Fmendoza-trees-workshop","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsimonw%2Fmendoza-trees-workshop/lists"}