https://github.com/eurec4a/eurec4a-intake

Intake catalogue for EUREC4A field campaign datasets
https://github.com/eurec4a/eurec4a-intake

Last synced: about 1 month ago
JSON representation

Intake catalogue for EUREC4A field campaign datasets

• Host: GitHub
• URL: https://github.com/eurec4a/eurec4a-intake
• Owner: eurec4a
• Created: 2020-07-27T18:08:39.000Z (over 4 years ago)
• Default Branch: master
• Last Pushed: 2024-05-01T17:47:15.000Z (8 months ago)
• Last Synced: 2024-10-30T04:45:17.264Z (2 months ago)
• Language: Python
• Size: 411 KB
• Stars: 17
• Watchers: 3
• Forks: 19
• Open Issues: 7
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • Contributing: CONTRIBUTING.md
  • Citation: CITATION.md

Awesome Lists containing this project

• open-sustainable-technology - EUREC4A Intake catalogue - Access to data from the EUREC4A field campaign, which aims to improve the understanding of the interplay between clouds, convection and circulation and their role in climate change. (Atmosphere / Atmospheric Chemistry and Aerosol)

README

        
# EUREC4A Intake catalogue

![weekly_tests.yml](https://github.com/eurec4a/eurec4a-intake/actions/workflows/weekly_tests.yml/badge.svg)

 [![](https://zenodo.org/badge/doi/10.5281/zenodo.8422321.svg)](https://doi.org/10.5281/zenodo.8422321)

This repository contains an [intake](https://github.com/intake/intake)

catalogue for acessing data from  the [EUREC4A field

campaign](http://eurec4a.eu/) stored on: 1)

[AERIS](https://observations.ipsl.fr/aeris/eurec4a/#/) and 2) Munich

University (via OPeNDAP) and 3) OPeNDAP access to files at

[NOAA's National Center for Environmental Information](https://www.ncei.noaa.gov/thredds-ocean/catalog/psl/atomic/catalog.html) and 4) data linked via [IPFS](https://ipfs.io/).

## Usage

To use you will need to install `intake`, `xarray`, `intake-xarray`, `zarr`,

`pydap`, `requests`, `s3fs` and `ipfsspec`

```bash

pip install "intake<2.0.0" xarray intake-xarray zarr pydap s3fs requests ipfsspec

```

**Or**, if you feel courageous (and want the newest updates), you can also install the [`requirements.txt`](requirements.txt) directly:

```bash

pip install -r https://raw.githubusercontent.com/eurec4a/eurec4a-intake/master/requirements.txt

```

The catalogue (and underlying data) can then be accessed directly from python:

```python

> from intake import open_catalog

> cat = open_catalog("https://raw.githubusercontent.com/eurec4a/eurec4a-intake/master/catalog.yml")

```

You can list the available sources with:

```python

>> list(cat)

['radiosondes', 'barbados', 'dropsondes', 'halo', 'p3', 'specmacs']

>> list(cat.radiosondes)

['atalante_meteomodem',

 'atalante_vaisala',

 'bco',

 'meteor',

 'ms_merian',

 'ronbrown']

```

Then load up a [dask](https://github.com/dask/dask)-backed `xarray.Dataset` so

that you have access to all the available variables and attributes in the

dataset:

```python

>> ds = cat.radiosondes.ronbrown.to_dask()

>> ds

Dimensions:      (alt: 3100, nv: 2, sounding: 329)

Coordinates:

  * alt          (alt) int16 0 10 20 30 40 50 ... 30950 30960 30970 30980 30990

    flight_time  (sounding, alt) datetime64[ns] dask.array

    lat          (sounding, alt) float32 dask.array

    lon          (sounding, alt) float32 dask.array

    sounding_id  (sounding) |S1000 dask.array

Dimensions without coordinates: nv, sounding

Data variables:

    N_gps        (sounding, alt) float32 dask.array

    N_ptu        (sounding, alt) float32 dask.array

    alt_bnds     (alt, nv) int16 dask.array

...

```

You can then slice and access the data as if you had it available locally

## Contributing

Please have a look at our [contribution guide](CONTRIBUTING.md).