{"id":19665586,"url":"https://github.com/fatiando/transform2020","last_synced_at":"2025-07-16T02:48:00.256Z","repository":{"id":40456792,"uuid":"259599690","full_name":"fatiando/transform2020","owner":"fatiando","description":"Material for the Verde tutorial at Transform 2020","archived":false,"fork":false,"pushed_at":"2022-09-24T05:40:53.000Z","size":13971,"stargazers_count":17,"open_issues_count":0,"forks_count":8,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-04-28T22:42:03.573Z","etag":null,"topics":["earth-science","fatiando-a-terra","geophysics","geoscience","geospatial","python","swung-t20"],"latest_commit_sha":null,"homepage":"https://transform2020.sched.com/event/c7KE/tutorial-from-scattered-data-to-gridded-products-using-verde","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fatiando.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-04-28T10:05:49.000Z","updated_at":"2024-01-15T14:57:17.000Z","dependencies_parsed_at":"2022-08-09T21:01:10.578Z","dependency_job_id":null,"html_url":"https://github.com/fatiando/transform2020","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/fatiando/transform2020","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fatiando%2Ftransform2020","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fatiando%2Ftransform2020/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fatiando%2Ftransform2020/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fatiando%2Ftransform2020/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fatiando","download_url":"https://codeload.github.com/fatiando/transform2020/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fatiando%2Ftransform2020/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265477387,"owners_count":23773046,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["earth-science","fatiando-a-terra","geophysics","geoscience","geospatial","python","swung-t20"],"created_at":"2024-11-11T16:23:33.948Z","updated_at":"2025-07-16T02:48:00.199Z","avatar_url":"https://github.com/fatiando.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# From scattered data to gridded products using Verde\n\nThe [Verde tutorial at Transform 2020](https://sched.co/c7KE) 💚\n\n**Instructors:**\n[Leonardo Uieda](https://www.leouieda.com/)\u003csup\u003e1\u003c/sup\u003e and\n[Santiago Soler](https://santisoler.github.io/)\u003csup\u003e2,3\u003c/sup\u003e\n\n\u003e \u003csup\u003e1\u003c/sup\u003eUniversity of Liverpool, UK |\n\u003e \u003csup\u003e2\u003c/sup\u003eCONICET, Argentina |\n\u003e \u003csup\u003e3\u003c/sup\u003eUniversidad Nacional de San Juan, Argentina\n\n|         | Info |\n|--------:|:-----|\n| When | Thursday, June 11 • 08:00 - 11:00 GMT |\n| Slack (Q\u0026A) | [Software Underground](https://softwareunderground.org/) channel `t20-thu-verde` |\n| Live stream | https://youtu.be/-xZdNdvzm3E |\n| conda environment  | `t20-thu-verde` |\n| Verde documentation | https://www.fatiando.org/verde |\n\n## BEFORE THE TUTORIAL\n\nMake sure you've done these things **before the tutorial on Thursday**:\n\n1. Sign-up for the [Software Underground Slack](https://softwareunderground.org/slack)\n1. Join the channel `t20-thu-verde`. This is where **all communication will\n   happend**.\n1. Set up your computer ([intructions below](#setup)). We will not have time to\n   solve many computer issues during the tutorial so make sure you do this\n   ahead of time. If you need any help, ask at the `t20-thu-verde` channel on\n   Slack.\n1. If you have some data you'd like to process, please have it ready and make\n   sure you can load it with pandas or numpy. You'll have some time at the end\n   of the tutorial to work on your own data.\n\n## About\n\nThis tutorial will be a hands-on tour of\n[Verde](https://www.fatiando.org/verde/), a Python package for processing and\ngridding geophysical/geospatial data with a twist of machine learning. We'll\nstart with a real dataset and work our way towards producing one or more\ngridded products. The way there will take us through:\n\n* Loading some data\n* Generating and handling coordinates and projections (using pyproj)\n* Splitting training and testing data for validation\n* Data decimation with blocked means/medians to avoid aliasing\n* 2D trend estimation\n* Gridding with bi-harmonic splines\n* Combining everything into a data processing pipeline\n* Cross-validation of data distributed spatially on the Earth (including\n  parallel execution with Dask)\n\n## Prerequisites\n\n* Some knowledge of Python is assumed (for example, you might want to attend\n  [this](https://transform2020.sched.com/event/c7Jm/getting-started-with-python) or\n  [this](https://transform2020.sched.com/event/c7Jn/more-python-for-subsurface) tutorial).\n* All coding will be done in Jupyter notebooks. I'll explain how they work\n  briefly but it will help if you've used them before.\n* We'll use [numpy](https://numpy.org/), [pandas](https://pandas.pydata.org/),\n  [xarray](http://xarray.pydata.org/), and [matplotlib](https://matplotlib.org/).\n  You don't need to be an expert in these tools but some familiarity will help.\n\n## Setup\n\nThere are a few things you'll need to follow the tutorial:\n\n1. A working Python intallation (Anaconda or Miniconda)\n2. The Verde tutorial *conda environment* installed\n3. A web browser that works with JupyterLab\n   (basically anything except Internt Explorer)\n\nTo get things setup, please do the following.\n\n**If you have any trouble**, please ask for help in the\n`t20-thu-verde` channel on the Software Underground slack.\n\n**Windows users:** When you see \"*terminal*\" in the instructions,\nthis means the \"*Anaconda Prompt*\" program for you.\n\n### Step 1\n\n**Follow the general instructions for Transform2020:** http://swu.ng/t20-python-setup\n(there are also YouTube videos of [Windows](https://youtu.be/FdatS_NKVrM)\nand [Linux](https://youtu.be/3ncwbHyZeAg))\n\nThis will get you a working Python 3 installation with the `conda` package\nmanager. If you already have one, you can skip this step.\n\n### Step 2\n\n**Create the `t20-thu-verde` conda environment:**\n\n1. Download the `environment.yml` file from [here](https://raw.githubusercontent.com/fatiando/transform2020/master/environment.yml) (right-click and select \"Save page as\" or similar)\n1. Make sure that the file is called `environment.yml`. Windows sometimes adds a\n   `.txt` to the end, which you should remove.\n1. Open a terminal. The following steps should be done in the terminal.\n1. Navigate to the folder that has the downloaded environment file\n   (if you don't know how to do this, take a moment to read [the Software\n   Carpentry lesson on the Unix shell](http://swcarpentry.github.io/shell-novice/)).\n1. Create the conda environment by running `conda env create --file environment.yml`\n   (this will download and install all of the packages used in the tutorial).\n\n### Step 3\n\n**Verify that the installation works:**\n\n1. Download the `test_install.py` script from [here](https://raw.githubusercontent.com/fatiando/transform2020/master/test_install.py)\n1. Open a terminal. The following steps should be done in the terminal.\n1. Activate the environment: `conda activate t20-thu-verde`\n1. Navigate to the folder where you downloaded `test_install.py`\n1. Run the test script: `python test_install.py`\n1. You should see this text in the terminal (the last part of the second line will depend\n   on your system):\n   ```\n   Verde version: v1.5.0\n   Downloading file 'baja-bathymetry.csv.xz' from 'https://github.com/fatiando/verde/raw/v1.5.0/data/baja-bathymetry.csv.xz' to '/home/USER/.cache/verde/v1.5.0'.\n   ```\n1. The following figure should pop up:\n\n[![Output of `test_python.py`.](https://raw.githubusercontent.com/fatiando/transform2020/master/test_install_output.png)](https://raw.githubusercontent.com/fatiando/transform2020/master/test_install_output.png)\n\nIf none of these commands gives an error, then your installation should be working.\nIf you get any errors or the outputs look significantly different,\nplease let us know on Slack at `#t20-thu-verde`.\n\n### Step 4\n\n**Start Jupyter notebook:**\n\n1. **Windows users:** Make sure you set a default browser that is **not Internet Explorer**. \n1. Activate the conda environment: `conda activate t20-thu-verde`\n1. Start the Jupyter notebook server: `jupyter notebook`\n1. Jupyter should open in your default web browser. We'll start from here in the\n   tutorial and create a new notebook together.\n\n### IF EVERYTHYNG ELSE FAILS\n\nIf you really can't get things to work on your computer,\nyou can run the code online through Google Colab (you will need a Google account).\nA starter notebook that installs Verde can be found here: https://swu.ng/ea31AO\n\nTo save a copy of the Colab notebook to your own account, click on the\n\"Open in playground mode\" and then \"Save to Drive\".\nYou might be interested in\n[this tutorial](https://transform2020.sched.com/event/c7Jn/tutorial-using-python-subsurface-tools-no-install-required)\nfor an overview of Google Colab.\n\n## How the tutorial will work\n\nDue to the number of participants, we can't do anything too interactive.\nInstead, we'll do a brief introduction, some live coding using an example\ndataset with a couple of exercises in the middle, and finally you'll have\ntime to work on your own data (or another example dataset).\n\n| Time          | Activity |\n|:-------------:|:---------|\n|  8:00 -  8:20 | Introduction |\n|  8:20 -  9:00 | Loading data, projections, slicing, blocked reductions, trends |\n|  9:00 -  9:20 | Break |\n|  9:20 - 10:00 | Train-test-split, gridding, cross-validation, hyper-parameter optimizationa |\n| 10:00 - 10:10 | Wrap-up: state of the project, future directions, how to get involved |\n| 10:10 - 11:00 | Q\u0026A on Slack, work on your own data, informal chat |\n\n## Data\n\nWe'll use some [public domain](https://www.ngdc.noaa.gov/ngdcinfo/privacy.html#disclaimer)\ndatasets in the tutorial. The datasets are trackline bathymetry measurements\nfrom the [NOAA NCEI](https://ngdc.noaa.gov/mgg/geodas/trackline.html).\nThe data files in [MGD77T format](https://ngdc.noaa.gov/mgg/dat/geodas/docs/mgd77.pdf)\nare stored in the `data` folder of this repository.\n\n| File | Location | Download | MD5 hash |\n| ---- | -------- | -------- | -------- |\n| `ew9105.m77t` | Wilkes Fracture Zone in the South Pacific | https://raw.githubusercontent.com/fatiando/transform2020/master/data/ew9105.m77t | `md5:27a0f501251b9dd5a1413722e1c406ea` |\n| `cd8794.m77t` | Reykjanes Ridge near Iceland | https://raw.githubusercontent.com/fatiando/transform2020/master/data/cd8794.m77t | `md5:48be7f63d020dacdb95116d88e35fc61` |\n| `b00297.m77t` | Loihi Seamount in Hawai'i | https://raw.githubusercontent.com/fatiando/transform2020/master/data/b00297.m77t | `md5:8a6eaadbd732b54f65075c9e18593f82` |\n\n## License\n\nAll code and text in this repository is free software: you can redistribute it and/or\nmodify it under the terms of the BSD 3-clause License.\nA copy of this license is provided in [LICENSE](https://github.com/fatiando/transform2020/blob/master/LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffatiando%2Ftransform2020","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffatiando%2Ftransform2020","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffatiando%2Ftransform2020/lists"}