{"id":18726902,"url":"https://github.com/pytroll/intake-satpy","last_synced_at":"2025-04-12T16:32:01.016Z","repository":{"id":136939471,"uuid":"579994137","full_name":"pytroll/intake-satpy","owner":"pytroll","description":"Intake drivers that use the Satpy package to read data","archived":false,"fork":false,"pushed_at":"2025-02-01T22:19:13.000Z","size":74,"stargazers_count":3,"open_issues_count":1,"forks_count":1,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-04-06T02:16:52.194Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pytroll.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2022-12-19T13:07:07.000Z","updated_at":"2024-12-02T15:47:16.000Z","dependencies_parsed_at":"2024-01-14T23:06:16.353Z","dependency_job_id":"3e74f136-4517-497b-9679-d49c675f0b8f","html_url":"https://github.com/pytroll/intake-satpy","commit_stats":{"total_commits":24,"total_committers":2,"mean_commits":12.0,"dds":0.375,"last_synced_commit":"090cd39b3e907d62564bea2ce34bb54bf7aa91c0"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pytroll%2Fintake-satpy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pytroll%2Fintake-satpy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pytroll%2Fintake-satpy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pytroll%2Fintake-satpy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pytroll","download_url":"https://codeload.github.com/pytroll/intake-satpy/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248596578,"owners_count":21130724,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T14:15:50.337Z","updated_at":"2025-04-12T16:32:00.590Z","avatar_url":"https://github.com/pytroll.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Intake - Satpy Drivers\n\nThis package adds additional drivers for the\n[Intake](https://intake.readthedocs.io/en/latest/) library using the\n[Satpy](https://satpy.readthedocs.io/en/stable/) library to read data files.\nThis package also depends on\n[intake-xarray](https://intake-xarray.readthedocs.io/en/latest/) to define\nthe Xarray container type (xarray `Dataset`) which these Satpy-based drivers\nproduce.\n\n## Installation\n\nTo add this package to an existing `pip` based environment, run:\n\n```bash\npip install intake-satpy\n```\n\nOr if you have a conda-based environment you can install it from the\nconda-forge channel:\n\n```bash\nconda install -c conda-forge intake-satpy\n```\n\n## Usage\n\nThis package currently only supplies one intake driver named `satpy`.\nAs with any intake driver, the `satpy` driver can be used in a couple\ndifferent ways. A few examples are shown below.\n\n### Inline Usage\n\nOnce the `intake-satpy` package is installed, you can use this driver by\ncalling `intake.open_satpy`. At the time of writing, it is best to provide\nas much information to configure/control Satpy as you can by passing the\n`scene_kwargs` and `load_kwargs`.\n\n```\nimport intake\nfrom glob import glob\n\ndata_source = intake.open_satpy(\n    glob(\"/data/satellite/abi/*.nc\"),\n    scene_kwargs={\"reader\": \"abi_l1b\"},\n    load_kwargs={\"wishlist\": [\"C01\"]},\n)\ndataset = data_source.read_chunked()\n```\n\nThe `read_chunked` method will return an xarray `Dataset` object that will\ncontain the products that Satpy was able to create. Data will be represented\nas dask arrays underneath. The `data_source.to_dask()` method will also\nproduce this result. The `data_source.read()` method will return the same\nxarray `Dataset` object but data will be loaded into memory as numpy arrays.\nCare must be taken as the large satellite formats read by Satpy can quickly\nfill up your system's memory if loaded in this way.\n\nBy default, if `wishlist` is not provided as a load keyword argument\n(see above), then all available \"reader\" level products will be loaded. This\nmeans those that can be read directly from the file and does not include\nany Satpy \"composites\".\n\nAlso by default the loaded dataset is \"resampled\" using Satpy's \"native\"\nresampler to the finest resolution of the loaded products. This allows\nfor all products to exist in a single xarray `Dataset` object. This behavior\ncan be customized by providing `resample_kwargs` to the source creation\n(`open_satpy` call).\n\n### Catalog Usage - Local\n\nThe `satpy` driver can also be used in a catalog definition. See the\n[examples/local_abi_l1b.yaml](https://github.com/pytroll/intake-satpy/blob/main/examples/local_abi_l1b.yaml)\ncatalog definition file for an example. With a catalog like this you could then\ndo:\n\n```python\nimport intake\n\ncat = intake.open_catalog(\"examples/local_abi_l1b.yaml\")\nsource = cat.abi_l1b(base_dir=\"/data/satellite/abi\")\ndataset = source.read_chunked()\n\n```\n\nA wishlist of products to load can be provided to the source when creating it:\n\n```python\ncat = intake.open_catalog(\"examples/local_abi_l1b.yaml\")\nsource = cat.abi_l1b(base_dir=\"/data/satellite/abi\", load_kwargs={\"wishlist\": [\"C01\"]})\ndataset = source.read_chunked()\n```\n\nAs with the inline usage, if `wishlist` is not provided then all reader-level\nproducts will be loaded.\n\n### Catalog Usage - S3\n\nSome of Satpy's readers can also read data from remote storage like S3 buckets.\nAn example catalog is included in the `examples/` directory of the\n`intake-satpy` repository.\n\nNote that Satpy's performance for reading S3 files is currently very slow, but\nis being worked on. It is likely not suitable for loading data outside of the\nnetwork where the S3 storage is (AWS in this example) until future updates to\nSatpy and NetCDF are made.\n\n```python\nimport intake\n\ncat = intake.open_catalog(\"examples/aws_abi_l1b_20220101_18.yaml\")\nsource = cat.abi_l1b()\ndataset = source.read_chunked()\n\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpytroll%2Fintake-satpy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpytroll%2Fintake-satpy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpytroll%2Fintake-satpy/lists"}