https://github.com/pytroll/fcidecomp
This is an fork to open for contributions from the community to the FCIDECOMP
https://github.com/pytroll/fcidecomp
Last synced: 9 months ago
JSON representation
This is an fork to open for contributions from the community to the FCIDECOMP
- Host: GitHub
- URL: https://github.com/pytroll/fcidecomp
- Owner: pytroll
- License: apache-2.0
- Created: 2024-01-31T10:16:49.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2024-01-31T12:26:11.000Z (over 2 years ago)
- Last Synced: 2025-05-19T19:19:14.785Z (about 1 year ago)
- Language: C
- Homepage: https://gitlab.eumetsat.int/open-source/data-tailor-plugins/fcidecomp
- Size: 21 MB
- Stars: 2
- Watchers: 5
- Forks: 0
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGES.md
- License: LICENSE
- Authors: AUTHORS.txt
Awesome Lists containing this project
README
# EUMETSAT ``fcidecomp`` software
The ``fcidecomp`` software enables decompression of near real-time spectral imagery data from the EUMETSAT Meteosat Third Generation (MTG) FCI instrument that are disseminated in compressed format (JPEG-LS netCDF). The decompression of files is possible via source code, netCDF-C and netCDF-Java based softwares, and various Python libraries
## Supported platforms and installation
The ``fcidecomp`` software can be installed on:
- Linux CentOS 7 64-bit
- RockyLinux 8 64-bit (also from source code)
- Linux Ubuntu 18.04 LTS 64-bit
- Linux Ubuntu 20.04 LTS 64-bit (also from source code)
- Windows 10 64-bit
- Windows 10 32-bit
## Installing the ``fcidecomp`` software
Installation procedures are described in the INSTALL file.
## Testing the ``fcidecomp`` software (only for the conda-based installation)
A set of Python unit tests is present to ensure the installed software works correctly. They should be run within the
Conda environment in which the software has been installed.
### Prerequisites
- `pytest`, installed in the Conda environment in which the software has been installed as described
[here](https://anaconda.org/anaconda/pytest)
Also, the tests depend on the presence of a set of test data, which can be downloaded
[here]().
Test files should be placed in a directories tree structured as follows (replace $EPCT_TEST_DATA_DIR
with any chosen name):
```BASH
|_$EPCT_TEST_DATA_DIR
|_MTG
|_MTGFCIL1
|_
|_
|_ ...
```
Once this is done, the environment variable `EPCT_TEST_DATA_DIR` should be set to the full path to the
`$EPCT_TEST_DATA_DIR` directory.
### Running the tests
Tests can be executed running the following command from within the root directory of the ``fcidecomp`` software repository:
pytest -vv tests
## Using the ``fcidecomp`` software
The ``fcidecomp`` decompression library can be used in different ways, described in the following sections.
### Use with netCDF4-C tools
### Prerequisites
To use ``fcidecomp`` with ``netCDF4-C``-based tools (e.g. ``nccopy``), two prerequisites are needed:
- the tools must be installed
- the ``HDF5_PLUGIN_PATH`` environment variable must be set as described in the `INSTALL` file.
If using the conda installation, simply activate the environment created above and the prerequisites are met.
If the package has been installed from the source code, set the environment variable and install the
relevant package (``netcdf-4.7.0`` on RockyLinux 8, ``netcdf-bin`` on Ubuntu Linux 20.04).
### Example with ``nccopy``
Once the prerequisites above are met, netCDF4-C tools should be automatically configured to decompress JPEG-LS
compressed netCDF files. As an example, to decompress a file using `nccopy`, run the following line:
nccopy -F none $PATH_TO_COMPRESSED_FILE $PATH_TO_DECOMPRESSED_FILE
where:
- `$PATH_TO_COMPRESSED_FILE` is the path to the JPEG-LS compressed file
- `$PATH_TO_DECOMPRESSED_FILE` is the path where the decompressed file should be saved
### Use with `h5py`-based Python libraries (conda install only)
Once the `fcidecomp` Conda package is installed and the Conda environment in which it is installed is activated,
use of the ``fcidecomp`` decompression libraries should be automatically enabled for `h5py`-based Python libraries.
To ensure the ``fcidecomp`` filter is loaded, in a Python shell execute:
import fcidecomp
Now every `h5py`-based Python library, such as `xarray`, will be able to open and read JPEG-LS compressed files without
further steps.
### Use with netCDF-Java based tools
With netCDF-Java versions greater than 5.5.2, it is possible to open JPEG-LS compressed netCDFs with netCDF-Java based
tools, such as toolsUI and Panoply, instructing netCDF-Java to use the netCDF-C library for reading purposes.
To enable this feature:
1. if ``fcidecomp`` has been installed from the source, install the netCDF4 library package (``netcdf-4.7.0``
on RockyLinux 8, ``libnetcdf-c++4`` on UbuntuLinux 20..04)
2. ensure the file `$HOME/.unidata/nj22Config.xml` exists (if it doesn't, it should be created) and
that it contains the following lines:
$PATH_TO_NETCDF_LIB_DIR
netcdf
true
where `$PATH_TO_NETCDF_LIB_DIR` is the path to the directory containing the `netcdf4` library, which:
- in Linux (conda install), corresponds to `$PATH_TO_CONDA_ENV/lib`
with `$PATH_TO_CONDA_ENV` equal to the path to the `conda` environment in which `fcidecomp` is installed.
- in Windows (conda install), corresponds to `$PATH_TO_CONDA_ENV\Library\lib`
with `$PATH_TO_CONDA_ENV` equal to the path to the `conda` environment in which `fcidecomp` is installed.
- in RockyLinux (install from source), corresponds to `/usr/lib64`
- in Ubuntu 20.04 (install from source), corresponds to `/usr/lib/x86_64-linux-gnu/`
Tested with ToolsUI 5.5.3 on Windows, Panoply 5.1.1 on Linux (known as not working for Panoply for that version in Windows due to a
Panoply issue).
Inventory Notices
-----------------
Licenses and copyright information for software dependencies up to version 2.0.0
is documented within the ``inventory`` folder.
Files listed under `inventory/items/data_proprietary.ABOUT` are licensed under EUMETSAT Proprietary license.
#### Dependencies
The following dependencies are not included in the package but are required and they will be downloaded at build or compilation time:
* component name, version, SPDX license id, copyright, home_url, comments
* charls, 2.1.0, BSD 3-Clause, - , https://github.com/team-charls/charls, - .
* hdf5, 1.10.*, BSD 3-Clause, - , https://www.h5py.org/, - .
* h5py, 2.* and 3.6.0, BSD 3-Clause, - , https://www.h5py.org/, - .
* python, 3.8.16, see https://docs.python.org/3/license.html, - , https://www.python.org/, - .
* zlib, 1.2.13, zlib (http://zlib.net/zlib_license.html), - , https://zlib.net/, - .
* libnetcdf, 4.8.1, MIT , - , https://www.unidata.ucar.edu/software/netcdf/, - .
* libssh2, 1.10.0, - , see https://www.libssh2.org/license.html , https://www.libssh2.org/, - .
* netcdf4, 1.6.2, -, - , https://unidata.github.io/netcdf4-python/, - .