Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/gregstarr/trough
main ionospheric trough data downloading and labeling
https://github.com/gregstarr/trough
data-science dmsp ionosphere magnetosphere satellite-data space-physics total-electron-content
Last synced: 3 days ago
JSON representation
main ionospheric trough data downloading and labeling
- Host: GitHub
- URL: https://github.com/gregstarr/trough
- Owner: gregstarr
- License: mit
- Created: 2020-06-08T21:01:20.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2024-08-14T19:37:51.000Z (4 months ago)
- Last Synced: 2024-11-02T10:11:28.534Z (about 2 months ago)
- Topics: data-science, dmsp, ionosphere, magnetosphere, satellite-data, space-physics, total-electron-content
- Language: Python
- Homepage:
- Size: 413 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 6
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# trough
[](https://github.com/gregstarr/trough/actions/workflows/python-package-conda.yml)
[](https://codecov.io/gh/gregstarr/trough)
### Example
### Features
- Download Madrigal TEC, OMNI and DMSP SSUSI data
- Process datasets into more convenient `xarray` data structures and save as NetCDF
- Automatically label main ionospheric trough
# Usage
1. Clone Repo
2. create conda environment: `conda env create -f environment.yml -n trough python=3.9`
3. `conda activate trough`
4. `pip install apexpy`
5. if you get a numpy error when you try to import apexpy: `pip install --upgrade nump`
6. install trough with `pip install -e .`
7. copy `config.json.example` --> `config.json` and change any options you want
8. run with `python -m trough config.json`
9. wait for it to finish (can take several days if you are running 5+ years)
10. add `import trough` in your code and access the data using `trough.get_data`
### Config
#### Main Options
| Config Option | Definition |
| --- |-------------------------------------------------------------------------------------------------------------------------|
| base_dir | base directory of trough downloads and processing, directories for downloading and processing will be created from here |
| madrigal_user_name | name supplied to MadrigalWeb |
| madrigal_user_email | email supplied to MadrigalWeb |
| madrigal_user_affil | affiliation supplied to MadrigalWeb |
| nasa_spdf_download_method | "http" or "ftp" (default) |
| lat_res | latitude resolution of processed TEC maps (degrees Apex magnetic latitude) |
| lon_res | longitude resolution of processed TEC maps (degrees Apex magnetic longitude) |
| time_res_unit | time resolution units (passed to `np.timedelta64`) |
| time_res_n | time resolution in units specified above (passed to `np.timedelta64`) |
| script_name | which script to run, available scripts are in `trough/scripts.py` |
| start_date | start date of interval (YYYYMMDD, YYYYMMDD_hh, YYYYMMDD_hhmm, or YYYYMMDD_hhmmss) |
| end_date | end date of interval, see "start_date" for format |
| keep_download | whether or not to keep the downloaded files (not recommended) |
| trough_id_params | trough labeling algorithm parameters, see below |
#### Trough Labeling Options
| Config Option | Definition |
| --- |-------------------------------------------------------------------------------------|
| bg_est_shape | background estimation filter size in pixels [time, latitude, longitude] |
| model_weight_max | maximum value of L2 regularization before multiplication by coefficient `l2_weight` |
| rbf_bw | RBF bandwidth, number of pixels to half weight |
| tv_hw | total variation horizontal weight |
| tv_vw | total variation vertical weight |
| l2_weight | L2 regularization coefficient |
| tv_weight | TV regularization coefficient |
| perimeter_th | minimum perimeter for a connected component in a label image |
| area_th | minimum area for a connected component in a label image |
| threshold | score threshold below which a pixel is not labeled as MIT |
| closing_rad | radius for disk structuring element passed to `skimage.morphology.binary_closing` |