{"id":19570368,"url":"https://github.com/vlouf/cpol_processing","last_synced_at":"2025-04-27T03:31:40.019Z","repository":{"id":118934016,"uuid":"102510925","full_name":"vlouf/cpol_processing","owner":"vlouf","description":"📡 Process meteorological radar data. 📡","archived":false,"fork":false,"pushed_at":"2021-09-06T09:03:58.000Z","size":9828,"stargazers_count":15,"open_issues_count":1,"forks_count":9,"subscribers_count":7,"default_branch":"master","last_synced_at":"2023-12-16T15:53:28.167Z","etag":null,"topics":["pyart","python","radar","rainfall","reflectivity"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vlouf.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2017-09-05T17:30:21.000Z","updated_at":"2023-12-20T16:33:31.177Z","dependencies_parsed_at":"2023-03-19T00:50:41.985Z","dependency_job_id":null,"html_url":"https://github.com/vlouf/cpol_processing","commit_stats":{"total_commits":567,"total_committers":5,"mean_commits":113.4,"dds":"0.22045855379188717","last_synced_commit":"097994422c46773754e04a3d4911b81c01673fa5"},"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vlouf%2Fcpol_processing","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vlouf%2Fcpol_processing/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vlouf%2Fcpol_processing/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vlouf%2Fcpol_processing/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vlouf","download_url":"https://codeload.github.com/vlouf/cpol_processing/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251085147,"owners_count":21533821,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["pyart","python","radar","rainfall","reflectivity"],"created_at":"2024-11-11T06:14:49.852Z","updated_at":"2025-04-27T03:31:39.728Z","avatar_url":"https://github.com/vlouf.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# Disclaimer\n\nThis dataset is supported by a funding from the U.S. Department of Energy as part of the Atmospheric Radiation Measurement (ARM) Climate Research Facility, an Office of Science user facility.\n\nIf you use this dataset to prepare a publication, please consider offering me (Valentin Louf) co-authorship and add the following line in the acknowledgments:\n\n\u003e This work has been supported by the U.S. Department of Energy Atmospheric Systems Research Program through the grant DE-SC0014063.\n\n# CPOL general information\n\n(Excerpt of my calibration paper)\n\n\u003e CPOL is a dual-polarization, Doppler radar, working at a frequency of 5.6 GHz with a pulse repetition frequency of 1000 Hz and a beamwidth of 1°. CPOL is located at Gunn Pt (-12.245°N, 131.045°E), about 25 km North-East from Darwin airport. CPOL performs a set of scans with an update time of 10 minutes. This includes, nominally, a volume scan, vertically pointing scan and two RHI scans. The scan comprises 15 elevations: 0.5°, 0.9°, 1.3°, 1.8°, 2.4°, 3.1°, 4.2°, 5.6°, 7.4°, 10°, 13.3°, 17.9°, 23.9°, 32°, and 43.1°. An additional series of scans at 90° is also performed regularly. The periodicity of the vertically pointing scan changes from season to season, and there are no such scans for seasons 2009/2010 and 2010/2011. The observed parameters are Z\u003csub\u003eh\u003c/sub\u003e, Z\u003csub\u003edr\u003c/sub\u003e, Doppler velocity (v), differential phase φ\u003csub\u003edp\u003c/sub\u003e, spectrum width σ\u003csub\u003ev\u003c/sub\u003e, and cross-correlation coefficient at zero lag ρ\u003csub\u003ehv\u003c/sub\u003e. The maximum unambiguous range of the volume scan is 140 km with a range gate spacing of 250 m and an azimuthal beam spacing of 1°. Between 2001 and 2007, to reduce the data size and allow real-time transmission to the regional forecasting office, the radar gate range was changed to 300 m, and data were sampled with an azimuthal resolution of 1.5°. Before 2007, the azimuthal indexing had to be corrected while, after 2007, the data are generated with the data synced to the azimuthal sampling. CPOL has produced more than 350,000 plan position indicator scans over 17 wet seasons (November to May) between 1998 to 2017. During that period, three seasons are missing: 2000/2001, 2007/2008, and 2008/2009. The first season is missing because the radar was moved to Sydney to support the 2000 Sydney Olympic games. The two latter seasons are missing because the radar antenna and receiver needed replacement.\n\nThe CPOL data are available at several levels. If you're not a radar guy, then you are probably interested in the level 2 data, as knowledge in radars and in signal processing are implicitly required to use the level 1 data. All levels use the netCDF format. The level 1 uses the CF/radial convention, while the level 2 tries to follow the CF convention.\n\n## Level 1a\n\nThe level 1a are for the radar peoples who want to play with the raw data. At this level, nothing has been processed, nothing has been removed (artifacts, noise, and clutter are presents), and nothing has passed through any quality check. Only use it if you know what you're doing. $\\rho_{hv}$ is missing from season 09/10, seasons after 2012 might contain a normalized coherent power field. The signal-to-noise ratio has to be computed for all seasons (radiosoundings are used for this).\n\nOn Raijin, the data are found here: `/g/data2/rr5/CPOL_radar/CPOL_level_1a`\n\n## Level 1b\n\nThese are the processed data. It comes in 3 versions:\n-\tPPI (radar coordinates, i.e. polar). On Raijin, it is found here: `/g/data2/rr5/CPOL_radar/CPOL_level_1b/PPI`\n-\tGRIDDED (In Cartesian coordinates). For the gridded data, \u003cb\u003ethe z-axis goes from 0 to 20 km of altitude by step of 0.5 km.\u003c/b\u003e\n    -\tGRID_150km_2500m: Full domain, with a 2.5 km grid resolution: `/g/data2/rr5/CPOL_radar/CPOL_level_1b/GRIDDED/GRID_150km_2500m`\n    -\tGRID_70km_1000m: Half the domain, with a 1 km grid resolution: `/g/data2/rr5/CPOL_radar/CPOL_level_1b/GRIDDED/GRID_70km_1000m`\n\nThe PPIs contain the following fields:\n-\tD0, median diameter\n-\tNW, intercept parameter\n-\tcorrected_differential_phase, corrected differential phase\n-\tcorrected_differential_reflectivity, corrected differential reflectivity (attenuation and range correction)\n-\tcorrected_specific_differential_phase, specific differential phase\n-\tcross_correlation_ratio, corrected cross correlation ratio (range correction).\n-\tdifferential_phase, raw differential phase\n-\tdifferential_reflectivity, raw differential reflectivity\n-\tradar_echo_classification, echo classification.\n-\tradar_estimated_rain_rate, rainfall rate (see appropriate citation down below)\n-\traw_velocity, velocity of scatterers away from instrument\n-\treflectivity, corrected reflectivity (noise and clutter remove, and attenuation corrected)\n-\tregion_dealias_velocity, Doppler velocity dealiased using the region based algorithm\n-\tsignal_to_noise_ratio, Signal to noise ratio, computed using radiosoundings from YPDN site.\n-\tspecific_attenuation_differential_reflectivity, differential reflectivity specific attenuation, using Bringi's coefficients.\n-\tspecific_attenuation_reflectivity, reflectivity specific attenuation, using Wu et al. relationship.\n-\tspectrum_width\n-\ttemperature, from radiosoundings\n-\ttotal_power, uncorrected reflectivity\n\nThe gridded data are made using the Barnes at al. algorithm from Py-ART with a constant radius of influence of 2.5 km or 1 km (depending on the grid resolution). The 1 km resolution data have these dimensions (z, y, x) -\u003e (41, 141, 141) and the 2.5 km resolution have these ones: (z, y, x) -\u003e (41, 117, 117). The z dimension goes from 0 to 20 km by step of 500 m.\n-\tD0\n-\tNW\n-\tcorrected_differential_reflectivity\n-\tcross_correlation_ratio\n-\tlatitude\n-\tlongitude\n-\tradar_echo_classification\n-\tradar_estimated_rain_rate\n-\treflectivity\n-\tregion_dealias_velocity\n-\ttemperature\n-\ttotal_power\n-\tvelocity\n\nIf you are interested in the gridding technique, please read Rob's post on github: [https://github.com/ARM-DOE/pyart/issues/753]\n\nQuicklooks and animation for the lowest elevation of the PPI data are available in the folders `FIGURE_CHECK` and `ANIMATED_GIF`, respectively.\n\n## Level 2\n\nThe level 2 are daily files, i.e. all the data for one day have been copied into a single file. Because the radar temporal resolution is 10 minutes, the time dimension length is 144 and it is set as constant in these files. These are 2D fields, but if 3D fields are a things of interest, then I'll consider adding them.\n\nThe data have constant dimensions:\n\n- (time, y, x) -\u003e (144, 117, 117) for the 2.5 km resolution files: `/g/data2/rr5/CPOL_radar/CPOL_level_2`\n- (time, y, x) -\u003e (144, 141, 141) for the 1 km resolution files: `/g/data2/rr5/CPOL_radar/CPOL_level_2_1km`\n\nThe following fields are available:\n- Reflectivity at 2500 m, copied from the level 1b gridded, in: `/g/data2/rr5/CPOL_radar/CPOL_level_2/REFLECTIVITY`\n- Rainfall rate, copied, and NaN values put to zero, from the level 1b gridded: `/g/data2/rr5/CPOL_radar/CPOL_level_2/RADAR_ESTIMATED_RAIN_RATE`\n- Steiner echo classification (0: nothing, 1: stratiform, 2: convective) at 2500 m, computed using the reflectivity at 2500m:\n`/g/data2/rr5/CPOL_radar/CPOL_level_2/STEINER_ECHO_CLASSIFICATION`\n- The 0dB echo top height, computed using Lakshman algorithm on the level 1B PPIs and then gridded manually, in: `/g/data2/rr5/CPOL_radar/CPOL_level_2/ECHO_TOP_HEIGHT`\n\n\n\u003cb\u003eIf you want more level 2 fields (that are already present on level 1b), like velocity, D0, $Z_{dr}$, or else, just ask for it ;-) \u003c/b\u003e\n\n\u003cb\u003e IMPORTANT:\u003c/b\u003e because we use a set of constant dimensions for all level 2 files, some time slices may be empty if no measurements have been made at this particular timestep. The variable called `isfile` will help you to know if measurements exist or not at a particular timestep. It is equal to 0 if there is no data and 1 if there is data.\n\n# Georeferencing information\n\nThe gridded product in level 1b contains 2D array of latitude/longitude. Here are the georeferencing information that you can find in the level 2 files metadata:\n\n```\n:geospatial_bounds = \"(129.70320368213441, 132.3856852067545, -13.552905831511362, -10.941777804922253)\" ;\n:geospatial_lat_min = \"-13.552905831511362\" ;\n:geospatial_lat_max = \"-10.941777804922253\" ;\n:geospatial_lat_units = \"degrees_north\" ;\n:geospatial_lon_min = \"129.70320368213441\" ;\n:geospatial_lon_max = \"132.3856852067545\" ;\n:geospatial_lon_units = \"degrees_east\" ;\n:geospatial_vertical_min = \"0\" ;\n:geospatial_vertical_max = \"20000\" ;\n:geospatial_vertical_resolution = \"500\" ;\n:geospatial_vertical_units = \"meters\" ;\n:origin_latitude = \"-12.249\" ;\n:origin_longitude = \"131.044\" ;\n:origin_altitude = \"50\" ;\n:geospatial_projection = \"Azimuthal equidistant projection\" ;\n```\n\nIn the next version of the CPOL level 2 data, these 2D lat/lon arrays will also be included.\n\n\n## Important note about rainfalls\n\nThe level 1b is a 3D fields, because that's how the algorithm that compute the rainfall rate works, BUT the rainfall rate is a surface field, it does not have meaning after the first elevation/first slice. If you were to use the level 1b, then only use the first elevation/slice of the `radar_estimated_rain_rate`.\n\nAt level 1b: the hydrometeors retreivals, namely D0, NW, and the rainfall rate at this level don't have any sort of post-processing applied to them, they are direct output from the algorithm that has been used to compute these fields. Meaning that \u003cb\u003eif the radar did not measure anything, then the rainfall rate is equal to NaN, not zero, even inside the radar domain.\u003c/b\u003e\n\nAt level 2: a post processing has been applied, the rainfalls has been put to zero instead of NaN. What is NaN is outside the CPOL domain. Inside the domain the rainfall rates have real values.\n\nMoreover, [Bobby Jackson](https://github.com/rcjackson) (Argonne Laboratory) recently worked CPOL rainfall rate retrievals and made comparisons with impact and video disdrometers. Here are his conclusion:\n\u003e The best estimate we have for rainfall so far is from the Thompson et al. blended technique .Thankfully, this is the one that Valentin already has in his processing code, so literally no work needs to be done to put that in the latest version. The only thing we are working on now is collaborating with Elizabeth Thompson on using her disdrometer data to estimate the retrieval random uncertainty by looking at the P.D.F. of rainfall rate as a function of the various moments and calculating the quartiles of the P.D.F at the given moments.\n\n## How to cite CPOL and/or the different products\n\nCPOL instrument:\n\n\u003e Keenan, T. et al. The BMRC/NCAR C-Band Polarimetric (C-POL) Radar System. J. Atmos. Ocean. Technol. 15, 871–886 (1998).\n\nCPOL calibration:\n\n\u003e Louf, V. et al. An integrated approach to weather radar calibration and monitoring using ground clutter and satellite comparisons. J. Atmos. Ocean. Technol. (Under review).\n\nThe level 1b were produced using the Py-ART software, it should be cited:\n\n\u003e Helmus, J. J. \u0026 Collis, S. M. The Python ARM Radar Toolkit (Py-ART ), a Library for Working with Weather Radar Data in the Python Programming Language. J. Open Res. Softw. 4, e25 (2016).\n\nThe following products are not \"raw measurements\" from the radar, they have been retrieved using various algorithm. If you use one of these products, the correct citations are:\n\nRainfall rate estimates:\n\n\u003e Thompson, E. J., Rutledge, S. A., Dolan, B., Thurai, M. \u0026 Chandrasekar, V. Dual-Polarization Radar Rainfall Estimation over Tropical Oceans. J. Appl. Meteorol. Climatol. 57, 755–775 (2018).\n\nSteiner echo classification:\n\n\u003e Steiner, M., Houze, R. A. \u0026 Yuter, S. E. Climatological Characterization of Three-Dimensional Storm Structure from Operational Radar and Rain Gauge Data. J. Appl. Meteorol. 34, 1978–2007 (1995).\n\nEcho top height:\n\n\u003e Lakshmanan, V., Hondl, K., Potvin, C. K. \u0026 Preignitz, D. An Improved Method for Estimating Radar Echo-Top Height. Weather Forecast. 28, 481–488 (2013).\n\nHere a recent paper were we verify the Echo top height:\n\n\u003e Jackson, R. C., Collis, S. M., Louf, V., Protat, A. \u0026 Majewski, L. A 17 year climatology of convective cloud top heights in Darwin. Atmos. Chem. Phys. Discuss. 1–26 (2018). doi:10.5194/acp-2018-408\n\nFor other products, like the dual-polarisation radar products, just contact me (I haven't listed them as I suppose very few people might know about them).\n\n# Known issues / future version\n\nLevel 2:\n- Only 1D arrays of latitude/longitude (the lat/lon crossing the radar origin) are available in the level 2, this should be fix in the next version. For now, use the georeferencing information in the metadata to rebuild the lat/lon arrays, or read the lat/lon arrays from the gridded level 1b files (next version).\n\nLevel 1b and potentially future level 2:\n- There are problems with the velocity dealiasing, due to a lack of \u003ci\u003e competent \u003c/i\u003e dealiasing algorithms. We developped a new and promising method of dealiasing that hopefully will solve this issue (next version).\n\nLevel 1b:\n- Processing of dual-polar products are always a struggle that is incrementally improved, the next version should use a more performing PHIDP/KDP processing.\n- The level 1b rainfall rates don't contain true zero values, but NaN values instead, it's not a bug, it's a feature (direct processing output). Use the level 2 rainfalls, or filled the array with zero and put to NaN all values outside of the radar domain.\n\n# Other libraries that use CPOL_processing\n\n- [PyHail](https://github.com/joshua-wx/PyHail)\n\n# Example with the level 1a\n\nLet's read, display information and plot some raw data.\n\n\n```python\n%matplotlib inline\nimport glob\nimport warnings\nimport netCDF4\nimport pyart\nimport numpy as np\nimport matplotlib.pyplot as pl\nimport dask\nimport dask.bag as db\nfrom dask.diagnostics import ProgressBar\nwarnings.simplefilter('ignore')\n```\n\n```python\nradar = pyart.io.read('/g/data2/rr5/CPOL_radar/CPOL_level_1a/2006/20060103/cfrad.20060103_001000.000_to_20060103_001816.000_CPOL_PPI_level1a.nc')\n```\n\n\n```python\nradar.info('compact')\n```\n\n    altitude: \u003cndarray of type: float64 and shape: (1,)\u003e\n    altitude_agl: None\n    antenna_transition: None\n    azimuth: \u003cndarray of type: float32 and shape: (4148,)\u003e\n    elevation: \u003cndarray of type: float32 and shape: (4148,)\u003e\n    fields:\n    \tDBZ: \u003cndarray of type: float64 and shape: (4148, 480)\u003e\n    \tVEL: \u003cndarray of type: float64 and shape: (4148, 480)\u003e\n    \tWIDTH: \u003cndarray of type: float64 and shape: (4148, 480)\u003e\n    \tZDR: \u003cndarray of type: float64 and shape: (4148, 480)\u003e\n    \tPHIDP: \u003cndarray of type: float64 and shape: (4148, 480)\u003e\n    \tRHOHV: \u003cndarray of type: float64 and shape: (4148, 480)\u003e\n    fixed_angle: \u003cndarray of type: float32 and shape: (17,)\u003e\n    instrument_parameters:\n    \tfrequency: \u003cndarray of type: float32 and shape: (1,)\u003e\n    \tpulse_width: \u003cndarray of type: float32 and shape: (4148,)\u003e\n    \tprt: \u003cndarray of type: float32 and shape: (4148,)\u003e\n    \tpolarization_mode: \u003cndarray of type: |S1 and shape: (17, 32)\u003e\n    \tradar_beam_width_h: \u003cndarray of type: float32 and shape: (1,)\u003e\n    \tradar_beam_width_v: \u003cndarray of type: float32 and shape: (1,)\u003e\n    latitude: \u003cndarray of type: float64 and shape: (1,)\u003e\n    longitude: \u003cndarray of type: float64 and shape: (1,)\u003e\n    nsweeps: 17\n    ngates: 480\n    nrays: 4148\n    radar_calibration: None\n    range: \u003cndarray of type: float32 and shape: (480,)\u003e\n    scan_rate: \u003cndarray of type: float32 and shape: (4148,)\u003e\n    scan_type: ppi\n    sweep_end_ray_index: \u003cndarray of type: int32 and shape: (17,)\u003e\n    sweep_mode: \u003cndarray of type: |S1 and shape: (17, 32)\u003e\n    sweep_number: \u003cndarray of type: int32 and shape: (17,)\u003e\n    sweep_start_ray_index: \u003cndarray of type: int32 and shape: (17,)\u003e\n    target_scan_rate: None\n    time: \u003cndarray of type: float32 and shape: (4148,)\u003e\n    metadata:\n    \tConventions: CF/Radial instrument_parameters\n    \tversion: 2017-10\n    \ttitle: PPI volume from Australia Bureau of Meteorology\n    \tinstitution: Australia Bureau of Meteorology\n    \treferences: If you use this dataset, please cite: 'An integrated approach to weather radar calibration and monitoring using ground clutter and satellite comparisons' by Louf et al. 2018 Journal of Atmospheric and Oceanic Technology.\n    \tsource:\n    \tcomment: This dataset has been created by Valentin Louf at the Bureau of Meteorology for Monash University and ARM.\n    \tinstrument_name: CPOL\n    \toriginal_container: UF\n    \tsite_name: Gunn_Pt\n    \tradar_name: CPOL\n    \tfield_names: DBZ VEL WIDTH ZDR PHIDP RHOHV\n    \tNCO: \"4.6.4\"\n    \tauthor: Valentin Louf\n    \tauthor_email: valentin.louf@bom.gov.au\n    \tcalibration: Calibration offsets applied: ZH = -1.2 dB, and ZDR = -0.13 dB.\n    \tcountry: Australia\n    \tcreated: 2017-10-26T13:04:17.512932\n    \tproject: CPOL\n    \tdescription: This dataset was processed using the ARM PyART toolbox, the trmm_rsl library, the NCO toolbox, and RadX from NCAR.\n    \tstate: NT\n    \thistory: October 2017 recalibration: Thu Aug  3 11:40:07 2017: ncrename -v .total_power,DBZ -v velocity,VEL -v spectrum_width,WIDTH -v differential_reflectivity,ZDR -v differential_phase,PHIDP -v cross_correlation_ratio,RHOHV -v .corrected_reflectivity,Refl cfrad.20060103_001000.000_to_20060103_001816.000_CPOL_PPI_level0.nc\n\n    \tvolume_number: 0\n    \tplatform_type: fixed\n    \tinstrument_type: radar\n    \tprimary_axis: axis_z\n\n\n\n```python\n# Note that Bathurst Island is missing from these plots, this is an issue coming from Basemap.\ngr = pyart.graph.RadarMapDisplay(radar)\nfig, ax = pl.subplots(3, 2, figsize=(12, 15), sharex=True, sharey=True)\nax = ax.flatten()\nsw = 0\ngr.plot_ppi_map('DBZ', ax=ax[0], sweep=sw, vmin=-35, vmax=65, cmap='pyart_NWSRef')\ngr.plot_ppi_map('ZDR', ax=ax[1], sweep=sw, vmin=-2, vmax=8, cmap='pyart_RefDiff')\ngr.plot_ppi_map('WIDTH', ax=ax[2], sweep=sw, vmin=0, vmax=30, cmap='pyart_NWS_SPW')\ngr.plot_ppi_map('RHOHV', ax=ax[3], sweep=sw, vmin=0.5, vmax=1.05, cmap='pyart_RefDiff')\ngr.plot_ppi_map('PHIDP', ax=ax[4], sweep=sw, cmap='pyart_Wild25', vmin=-180, vmax=180)\ngr.plot_ppi_map('VEL', ax=ax[5], sweep=sw, cmap='pyart_NWSVel')\n\nfor ax_sl in ax:\n    gr.plot_range_rings([50, 100, 150], ax=ax_sl)\n    ax_sl.set_aspect(1)\n\npl.show()\n```\n\n\n![png](output_11_0.png)\n\n\n# Example with the level 1b\n\nRead all files during a day and plot some QVPs.\n\n\n```python\ndef retrieve_qvp(filename, fields=None, desired_angle=20.0):\n    if fields == None:\n        fields = ['differential_phase', 'cross_correlation_ratio', 'total_power', 'differential_reflectivity']\n\n    try:\n        radar = pyart.io.read(filename)\n    except Exception:\n        return None\n    index = abs(radar.fixed_angle['data'] - desired_angle).argmin()\n\n    qvp = {}\n    for field in fields:\n        data = radar.get_field(index, field).mean(axis = 0)\n        qvp.update({field: data})\n\n    qvp.update({'range': radar.range['data'], 'time': radar.time})\n    z = qvp['range'] / 1000.0 * np.sin(radar.fixed_angle['data'][index] * np.pi / 180)\n    qvp.update({'height': z})\n\n    del radar\n    return qvp\n```\n\n\n```python\nflist = sorted(glob.glob('/g/data2/rr5/CPOL_radar/CPOL_level_1b/PPI/2017/20170304/*.nc'))\n```\n\n\n```python\nbag = db.from_sequence(flist)\ndbobj = bag.map(retrieve_qvp)\nwith ProgressBar():\n    rslt = dbobj.compute()\n\nrslt = [r for r in rslt if r is not None]\n```\n\n    [########################################] | 100% Completed | 41.8s\n\n\n\n```python\n# Unpack data\ndifferential_phase = np.zeros((599, len(rslt)))\ncross_correlation_ratio = np.zeros((599, len(rslt)))\nreflectivity = np.zeros((599, len(rslt)))\ndifferential_reflectivity = np.zeros((599, len(rslt)))\ntime = [None] * len(rslt)\n\nfor i, r in enumerate(rslt):\n    differential_phase[:, i] = r['differential_phase']\n    cross_correlation_ratio[:, i] = r['cross_correlation_ratio']\n    reflectivity[:, i] = r['total_power']\n    differential_reflectivity[:, i] = r['differential_reflectivity']\n    time[i] = netCDF4.num2date(r['time']['data'][0], r['time']['units'])\nheight = r['height']\n```\n\n\n```python\ntitles = ['Reflectivity (dBZ)', r'$Z_{dr}$ (dB)', r'$\\phi_{dp}\\,(^\\circ)$', r'$\\rho_{hv}$', r'$\\omega$ (m/s)']\nwith pl.style.context('bmh'):\n    fig, ax = pl.subplots(4, 1, figsize=(10, 12), sharex=True)\n    ax = ax.flatten()\n    cax = [None] * len(ax)\n\n    cax[0] = ax[0].pcolormesh(time, height, reflectivity, cmap='pyart_NWSRef', vmin=-15, vmax=75)\n    ax[0].set_title('Evolution of tropical cyclone Blanche\\nCPOL 2017-03-04')\n    cax[1] = ax[1].pcolormesh(time, height, differential_reflectivity, cmap='pyart_RefDiff', vmin=-1, vmax=8)\n    cax[2] = ax[2].pcolormesh(time, height, 90 + differential_phase, cmap='pyart_Wild25', vmin=-180, vmax=180)\n    cax[3] = ax[3].pcolormesh(time, height, cross_correlation_ratio, cmap='pyart_RefDiff', vmin=0.5, vmax=1.05)\n\n    for i in range(len(ax)):\n        ax[i].set_ylim(0, 20)\n        ax[i].set_ylabel('Height (km)')\n        cbar = pl.colorbar(cax[i], ax=ax[i])\n        cbar.set_label(titles[i])\n\n    fig.tight_layout()\npl.show()\n```\n\n\n![png](output_17_0.png)\n\n\n# Example with the level 2\n\nIn this example we remake the Fig. 3 from Kumar et al. (2013) \u003ci\u003e The four cumulus cloud modes and their progression during rainfall events: A C-band polarimetric radar perspective \u003c/i\u003e.\n\n\n```python\ndef read_data(infile, key='radar_estimated_rain_rate'):\n    with netCDF4.Dataset(infile) as ncid:\n        data = np.squeeze(ncid[key][:])\n    return data\n```\n\n\n```python\ndef proc_file(inargs):\n    file_eth, file_stein, file_refl = inargs\n\n    eth = read_data(file_eth, 'echo_top_height')\n    stein = read_data(file_stein, 'steiner_echo_classification')\n    refl = read_data(file_refl, 'reflectivity')\n    pos = stein == 2\n\n    return eth[pos], refl[pos]\n```\n\n\n```python\nflist_eth = sorted(glob.glob('/g/data2/rr5/CPOL_radar/CPOL_level_2/ECHO_TOP_HEIGHT/*.nc'))\nflist_stein = sorted(glob.glob('/g/data2/rr5/CPOL_radar/CPOL_level_2/STEINER_ECHO_CLASSIFICATION/*.nc'))\nflist_refl = sorted(glob.glob('/g/data2/rr5/CPOL_radar/CPOL_level_2/REFLECTIVITY/*.nc'))\narg_list = [(a, b, c) for a, b, c in zip(flist_eth, flist_stein, flist_refl)]\n```\n\n\n```python\nbag = db.from_sequence(arg_list)\nbagobj = bag.map(proc_file)\nwith ProgressBar():\n    rslt = bagobj.compute()\n```\n\n    [########################################] | 100% Completed | 42.7s\n\n\n\n```python\n# Unpacking results\nct = np.sum([len(a) for a, b in rslt])\neth = np.zeros((ct))\nrefl = np.zeros((ct))\npos = 0\nfor a, b in rslt:\n    if len(a) == 0:\n        continue\n    eth[pos: pos + len(a)] = a\n    refl[pos: pos + len(a)] = b\n    pos += len(a)\n\npos = refl \u003e 0\neth = eth[pos]\nrefl = refl[pos]\n```\n\n\n```python\nwith pl.style.context('bmh'):\n    pl.hist2d(refl, eth, range=[[10, 60], [3e3, 20e3]], bins=[50, 40], cmap='jet')\n    pl.xlabel('Reflectivity at 2.5 km (dBZ)')\n    pl.ylabel('0dB Echo Top Height (m)')\n    pl.hlines(6.5e3, 10, 60, 'w', 'dashed')\n    pl.hlines(15e3, 10, 60, 'w', 'dashed')\n    pl.title('CPOL 1998-2017.\\n{:,} elements.'.format(ct))\n    pl.colorbar()\npl.show()\n```\n\n\n![png](output_24_0.png)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvlouf%2Fcpol_processing","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvlouf%2Fcpol_processing","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvlouf%2Fcpol_processing/lists"}