{"id":16877420,"url":"https://github.com/juntyr/lorenz96-compress","last_synced_at":"2025-03-19T06:03:43.927Z","repository":{"id":200939353,"uuid":"706540602","full_name":"juntyr/lorenz96-compress","owner":"juntyr","description":"Exploring online compression with the Lorenz96 model","archived":false,"fork":false,"pushed_at":"2023-11-27T10:49:02.000Z","size":659855,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-01T04:37:29.449Z","etag":null,"topics":["compression","esiwace","hackathon","lorenz96","zfp"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/juntyr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-10-18T06:43:23.000Z","updated_at":"2023-11-26T16:25:17.000Z","dependencies_parsed_at":"2023-11-24T23:22:28.874Z","dependency_job_id":"47c1173a-7305-4a9a-80d2-3353c8393ee7","html_url":"https://github.com/juntyr/lorenz96-compress","commit_stats":null,"previous_names":["juntyr/lorenz96-compress"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/juntyr%2Florenz96-compress","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/juntyr%2Florenz96-compress/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/juntyr%2Florenz96-compress/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/juntyr%2Florenz96-compress/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/juntyr","download_url":"https://codeload.github.com/juntyr/lorenz96-compress/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244365068,"owners_count":20441472,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["compression","esiwace","hackathon","lorenz96","zfp"],"created_at":"2024-10-13T15:43:30.368Z","updated_at":"2025-03-19T06:03:43.896Z","avatar_url":"https://github.com/juntyr.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Exploring online compression with the Lorenz96 model \u0026emsp; [![DOI]][doi-url]\n- - - -\n\n[DOI]: https://zenodo.org/badge/DOI/10.5281/zenodo.10207094.svg\n[doi-url]: https://doi.org/10.5281/zenodo.10207094\n\nThis project was developed during the ESiWACE3 hackathon at CSC Finland on 18.10. - 20.10.2023 by Victor Azizi from the Netherlands eScience Center and Juniper Tyree from the University of Helsinki.\n\n## Compiling ZFP for LUMI-G\n\nThis sections details the process to compile ZFP on LUMI-G. Note that the Makefile should take care of the entire process - the following are Victor Azizi's notes on how he got ZFP to compile:\n\n- First, check out the zfp submodule:\n  ```bash\n  git submodule init\n  git submodule update\n  ```\n  Alternatively, clone the repository and check out the staging branch (checked with hash e935a81):\n  ```bash\n  git clone https://github.com/LLNL/zfp.git\n  cd zfp\n  git checkout staging\n  ```\n- Next, load the right modules on LUMI:\n  ```bash\n  module load LUMI\n  module load buildtools\n  ```\n  They are also available in the `modules` file, which can be `source`d.\n- Now, configure the cmake build of ZFP:\n  ```bash\n  mkdir -p build\n  cd build\n  cmake -DZFP_WITH_HIP=ON -DZFP_WITH_OPENMP=OFF -DHIP_PATH=/opt/rocm/hip -DCMAKE_C_COMPILER=hipcc -DCMAKE_CXX_COMPILER=hipcc -DBUILD_TESTING=OFF ../\n  ```\n- Next, build with VERBOSE, as you will manually have to fix some commands\n  ```bash\n  make VERBOSE=1\n  ```\n- You now have a library, but the compilation step of `zfp.dir/hip/interface.cpp.o` shows that the wrong offload target was used. Therefore, you have to manually fix the offload target to `gfx90a`:\n  ```bash\n  cd build\n  CC -DZFP_ROUNDING_MODE=ZFP_ROUND_NEVER -DZFP_WITH_HIP -Dzfp_EXPORTS -I../../include -O3 -DNDEBUG --offload-arch=gfx90a -fPIC -std=gnu++14 -o CMakeFiles/zfp.dir/hip/interface.cpp.o -x hip -c ../../src/hip/interface.cpp\n  ```\n- Finally, rebuild the library:\n  ```bash\n  cd build\n  make\n  ```  \n- Success! The AMD GPU enabled library can now be found in `zfp/build/lib64/libzfp.so`\n\n## Reproducing the plot notebooks from the command line\n\n- First, clear the previous output:\n  ```bash\n  jupyter nbconvert --clear-output --inplace plot.ipynb\n  ```\n- Second, use `nbconvert` or `papermill` to execute the notebook, e.g.:\n  ```bash\n  papermill --kernel python3 --autosave-cell-every 5 --progress-bar plot.ipynb plot.nbconvert.ipynb\n  ```\n- Optionally, remove execution metadata:\n  ```bash\n  jupyter nbconvert --ClearMetadataPreprocessor.enabled=True --to=notebook --inplace plot.nbconvert.ipynb\n  ```\n- Finally, replace the old notebook with the new:\n  ```bash\n  mv plot.nbconvert.ipynb plot.ipynb\n  ```\n\n## License\n\nUnless specified otherwise below, this repository is licensed under either of\n\n * Apache License, Version 2.0\n   ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)\n * MIT license\n   ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)\n\nat your option.\n\n---\n\n- The [`cmdparser.hpp`](cmdparser.hpp) file, which is part of Florian Rappl's [C++ CmdParser utility](https://github.com/FlorianRappl/CmdParser), is licensed under the MIT license ([`FlorianRappl/CmdParser/LICENSE`](https://github.com/FlorianRappl/CmdParser/blob/master/LICENSE)).\n- The [`output/`](output/) folder, which contains the experiment output files, is licensed under the CC0 1.0 license ([`output/LICENSE`](output/LICENSE) or https://creativecommons.org/publicdomain/zero/1.0/).\n- The [`performance/`](performance/) folder, which contains the experiment performance files, is licensed under the CC0 1.0 license ([`performance/LICENSE`](performance/LICENSE) or https://creativecommons.org/publicdomain/zero/1.0/).\n- The [`zfp/`](https://github.com/LLNL/zfp/tree/e935a81d716a9e1d464ed214ed3c1c28157259ec) submodule, is licensed under the BSD 3-Clause license ([`zfp/LICENSE`](https://github.com/LLNL/zfp/blob/e935a81d716a9e1d464ed214ed3c1c28157259ec/LICENSE) and [`zfp/NOTICE`](https://github.com/LLNL/zfp/blob/e935a81d716a9e1d464ed214ed3c1c28157259ec/NOTICE)).\n\n## Citation\n\nPlease refer to the [CITATION.cff](CITATION.cff) file and refer to https://citation-file-format.github.io to extract the citation in a format of your choice.\n\n## Funding\n\nThis project has been developed as part of [ESiWACE3](https://www.esiwace.eu), the third phase of the Centre of Excellence in Simulation of Weather and Climate in Europe.\n\nFunded by the European Union. This work has received funding from the European High Performance Computing Joint Undertaking (JU) under grant agreement No 101093054.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjuntyr%2Florenz96-compress","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjuntyr%2Florenz96-compress","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjuntyr%2Florenz96-compress/lists"}