{"id":15115789,"url":"https://github.com/mit-ll/tornet","last_synced_at":"2026-04-07T17:33:31.092Z","repository":{"id":219324693,"uuid":"747304793","full_name":"mit-ll/tornet","owner":"mit-ll","description":"Software to work with the TorNet dataset","archived":false,"fork":false,"pushed_at":"2025-05-27T17:21:44.000Z","size":91993,"stargazers_count":60,"open_issues_count":7,"forks_count":13,"subscribers_count":10,"default_branch":"main","last_synced_at":"2025-05-27T18:30:48.204Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mit-ll.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-01-23T17:07:35.000Z","updated_at":"2025-05-27T17:21:48.000Z","dependencies_parsed_at":"2024-07-12T17:53:43.785Z","dependency_job_id":"2bf540ba-983e-4133-91fe-867cc1571c8f","html_url":"https://github.com/mit-ll/tornet","commit_stats":null,"previous_names":["mit-ll/tornet"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/mit-ll/tornet","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mit-ll%2Ftornet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mit-ll%2Ftornet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mit-ll%2Ftornet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mit-ll%2Ftornet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mit-ll","download_url":"https://codeload.github.com/mit-ll/tornet/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mit-ll%2Ftornet/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31522424,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-07T16:28:08.000Z","status":"ssl_error","status_checked_at":"2026-04-07T16:28:06.951Z","response_time":105,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-09-26T01:44:01.076Z","updated_at":"2026-04-07T17:33:31.063Z","avatar_url":"https://github.com/mit-ll.png","language":"Jupyter Notebook","funding_links":[],"categories":["Jupyter Notebook"],"sub_categories":[],"readme":"# TorNet\n\nSoftware to work with the TorNet dataset as described in the paper [*A Benchmark Dataset for Tornado Detection and Prediction using Full-Resolution Polarimetric Weather Radar Data*](https://journals.ametsoc.org/view/journals/aies/4/1/AIES-D-24-0006.1.xml)\n\n## Updates (7/9/24)\n\n* The TorNet dataset has been updated to version 1.1.  This update fixes a small number of incorrectly labeled frames in v1 of the dataset, and fixes the event and episode IDs of the warning categoies.   Version 1.1 also provides the tornado start and end times in the metadata.  We recommend re-downloading the newer version of the data using the links below. \n\n* The code and pretrained models are now compatible with `keras` 3.0.   Users can now select their deep learning backend from `tensorflow`, `torch`, or `jax`.  Backend-agnostic data loaders are also provided.  Read more about this library at [keras's website](https://keras.io/).  Users of `tf.keras` should use the `tf_keras` branch of this repo.  \n\n* The pretrained CNN model is now available on [huggingface (tornet-ml/tornado_detector_baseline_v1)](https://huggingface.co/tornet-ml/tornado_detector_baseline_v1).   Instructions for downloading and using the pre-trained model can be found in `models/README.md` and in the `VisualizeSamples.ipynb` notebook.\n\n\n![Alt text](tornet_image.png?raw=true \"sample\")\n\n\n\n## Downloading the Data\n\nThe TorNet dataset can be downloaded from the following location:\n\n#### Zenodo\n\nTorNet is split across 10 files, each containing 1 year of data. There is also a catalog CSV file that is used by some functions in this repository.    \n\n* Tornet 2013 (3 GB) and catalog: [https://doi.org/10.5281/zenodo.12636522](https://doi.org/10.5281/zenodo.12636522)\n* Tornet 2014 (15 GB): [https://doi.org/10.5281/zenodo.12637032](https://doi.org/10.5281/zenodo.12637032)\n* Tornet 2015 (17 GB): [https://doi.org/10.5281/zenodo.12655151](https://doi.org/10.5281/zenodo.12655151)\n* Tornet 2016 (16 GB): [https://doi.org/10.5281/zenodo.12655179](https://doi.org/10.5281/zenodo.12655179)\n* Tornet 2017 (15 GB): [https://doi.org/10.5281/zenodo.12655183](https://doi.org/10.5281/zenodo.12655183)\n* Tornet 2018 (12 GB): [https://doi.org/10.5281/zenodo.12655187](https://doi.org/10.5281/zenodo.12655187)\n* Tornet 2019 (18 GB): [https://doi.org/10.5281/zenodo.12655716](https://doi.org/10.5281/zenodo.12655716)\n* Tornet 2020 (17 GB): [https://doi.org/10.5281/zenodo.12655717](https://doi.org/10.5281/zenodo.12655717)\n* Tornet 2021 (18 GB): [https://doi.org/10.5281/zenodo.12655718](https://doi.org/10.5281/zenodo.12655718)\n* Tornet 2022 (19 GB): [https://doi.org/10.5281/zenodo.12655719](https://doi.org/10.5281/zenodo.12655719)\n\nIf downloading through your browser is slow, we recommend downloading these using `zenodo_get` (https://gitlab.com/dvolgyes/zenodo_get).\n\nAfter downloading, there should be 11 files, `catalog.csv`, and 10 files named as `tornet_YYYY.tar.gz`.   Move and untar these into a target directory, which will be referenced using the `TORNET_ROOT` environment variable in the code.  After untarring the 10 files, this directory should contain `catalog.csv` along with sub-directories `train/` and `test/` filled with `.nc` files for each year in the dataset.\n\n\n## Setup\n\nBasic python requirements are listed in `requirements/basic.txt`.\n\nThe `tornet` package can then installed into your environment by running\n\n`pip install .`\n\nIn this repo.  To do ML with TorNet, additional installs may be necessary depending on library of choice.  See e.g., `requirements/tensorflow.txt`, `requirements/torch.txt` and/or `requirements/jax.txt`.  \n\nPlease note that we did not exhaustively test all combinations of operating systems, data loaders, deep learning frameworks, and GPU usage.  If you are using the latest version of `keras`, then I recommend you follow setup instructions on the keras webpage [https://keras.io/getting_started/](https://keras.io/getting_started/).  Feel free to describe any issues you are having under the issues tab.  \n\n### Conda\n\nIf using conda\n\n```\nconda create -n tornet-{backend} python=3.10\nconda activate tornet-{backend}\npip install -r requirements/{backend}.txt\n```\n\nReplace {backend} with tensorflow, torch or jax.\n\n\n## Loading and visualizing TorNet\n\nStart with `notebooks/DataLoaders.ipynb` to get an overview on loading and visualizing the dataset.\n\nTo run inference on TorNet samples using a pretrained model,  look at `notebooks/VisualizeSamples.ipynb`.\n\n## Train CNN baseline model\n\n### Multiple backend support with Keras 3\nThe model uses Keras 3 which supports multiple backends. The environment variable\nKERAS_BACKEND can be used to choose the backend. \n\n```\nexport KERAS_BACKEND=tensorflow\n# export KERAS_BACKEND=torch\n# export KERAS_BACKEND=jax\n```\n\nThe following trains the CNN baseline model described in the paper using `tensorflow`.  If you run this out-of-the-box, it will run very slowly because it uses the basic dataloader.  Read the DataLoader notebook for tips on how to optimize the data loader.\n```\n# Set path to dataset\nexport TORNET_ROOT=/path/to/tornet     \n\n# Run training\npython scripts/tornado_detection/train_tornado_keras.py scripts/tornado_detection/config/params.json\n```\n\n## Evaluate trained model\nTo evaluate this model on the test set, run\n\n```\n# Set path to dataset\nexport TORNET_ROOT=/path/to/tornet  \n\n# Evaluate trained model\npython scripts/tornado_detection/test_tornado_keras.py \n```\n\nThis will compute and print various metrics computed on the test set.  Note that this script will attempt to download pretrained weights from huggingface, so ensure there is internet connectivity.  Alternatively, manually download the pretrained yourself and provide with `--model_path`\n\n\n### Disclosure\n```\nDISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.\nThis material is based upon work supported by the Department of the Air Force under Air Force Contract No. FA8702-15-D-0001. Any opinions, findings, conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the Department of the Air Force.\n© 2024 Massachusetts Institute of Technology.\nThe software/firmware is provided to you on an As-Is basis\nDelivered to the U.S. Government with Unlimited Rights, as defined in DFARS Part 252.227-7013 or 7014 (Feb 2014). Notwithstanding any copyright notice, U.S. Government rights in this work are defined by DFARS 252.227-7013 or DFARS 252.227-7014 as detailed above. Use of this work other than as specifically authorized by the U.S. Government may violate any copyrights that exist in this work.\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmit-ll%2Ftornet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmit-ll%2Ftornet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmit-ll%2Ftornet/lists"}